Polarity Checker

Overview

Resources

• Full write-up (Jupyter notebook)

Arlington 2050: the source data

• Arlington 2050 is a community planning project where residents shared what they think Arlington will look like in 2050.
• I worked specifically with responses collected at the county fair.
• With that many free-text responses, I used code to turn them into structured data and visualizations.

The process

Everything ran through Python, using pandas to move data between Excel files and code.

import pandas as pd

Load the survey export and assign it to a variable:

array1 = pd.read_excel("CountyFair.xlsx")

Rename the columns so they're easier to work with:

ds = array1.rename(columns={
  "Unnamed: 1": "Year2050",
  "Unnamed: 2": "Translation1",
  "Unnamed: 3": "Getting_Here",
  "Unnamed: 4": "Translation2"
})

From there, spaCy handles the NLP side:

import spacy
from wordcloud import WordCloud, STOPWORDS
import matplotlib.pyplot as plt
from spacytextblob.spacytextblob import SpacyTextBlob

Some responses came in Spanish. This pass swaps in the English translation column wherever one exists:

nlp = spacy.load('en_core_web_sm')
nlp.add_pipe('spacytextblob')

string_list = ds['Year2050'].tolist()
spanish_list = ds['Translation1'].tolist()
IndexCounter = 0
for n in spanish_list:
    workingstring = str(n)
    if workingstring != 'nan':
        string_list[IndexCounter] = workingstring
    IndexCounter += 1

Building a word cloud

First, pull every word out of the responses, excluding stop words and punctuation:

text = ds['Year2050'].str.cat(sep='')
doc = nlp(text)
words = [token.text for token in doc if not token.is_stop and not token.is_punct]

Then generate the cloud itself:

wordcloud = WordCloud(width=800, height=400, background_color='white',
                      max_words=100, contour_width=3, contour_color='steelblue'
                     ).generate(" ".join(words))

plt.figure(figsize=(10, 5))
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis("off")
plt.show()

The result surfaces the survey's major themes at a glance — housing, parks, and community came up the most.

Polarity and subjectivity histograms

Next, compute polarity and subjectivity for every response:

pol_list = []
sub_list = []
for t in range(2, len(string_list)):
    text = string_list[t]
    doc = nlp(text)
    pol_list.append(doc._.blob.polarity)
    sub_list.append(doc._.blob.subjectivity)

With the plotting libraries in place:

import seaborn as sns
import numpy as np

Subjectivity

plt.figure(figsize=(10, 6))
sns.histplot(data=sub_list)
plt.title('Subjectivity of Postcard Responses from County Fair')
plt.xlabel('In a range from 0 to 1')
plt.ylabel('Frequency')
plt.grid(True)
plt.show()

Polarity

plt.figure(figsize=(10, 6))
sns.histplot(data=pol_list)
plt.title('Polarity of Postcard Responses from County Fair')
plt.xlabel('In a range from -1 to 1')
plt.ylabel('Frequency')
plt.grid(True)
plt.show()

Related techniques worth knowing

Dimensionality reduction

• Simplifies data by reducing the number of features.
• Helps eliminate redundancy and noise in the model.

Vector embeddings

• Vector embeddings map words to numerical vectors.
• They let you mathematically compare word similarity, or detect outliers.

An example, checking vector stats for common words versus nonsense strings:

import spacy
nlp = spacy.load("en_core_web_lg") 
tokens = nlp("dog cat banana afskfsd")

for token in tokens:
    print(token.text, token.has_vector, token.vector_norm, token.is_oov)

Summary

Some of what's covered here is genuinely college-level material, but the Arlington 2050 project made it concrete: it's a real introduction to how language gets represented mathematically, which is a core part of how AI systems interpret text. It was a solid first pass at natural language processing and real-world data visualization.