Sentiment analysis is a natural language processing (NLP) technique used to determine the emotional tone or attitude expressed in text. Text responses are placed into categories, such as positive, negative, or neutral. Or even more specific domains, like belonging or impact on learning.
Sentiment analysis is relevant to analyzing survey data because the number of students responding to open-ended survey questions has grown significantly over the last 10 years.
I used a desktop application (Microsoft Power Query) to create sentiment scores based on the most recent open ended NSSE survey question: What one change would most improve the educational experience at ISU, and what one thing should not be changed?
By associating sentiment scores with survey data, I found that NLP model does a decent job of assigning sentiment scores.
However, the NLP model detected different sentiments for seniors than first-year students. The example of belonging is below.
And the sentiment scores added little to the NSSE retention logistic regression model.
Implications for Evaluation & Assessment
- Sentiment scores are better at explaining than predicting, at least in Microsoft Power Query.
- Sentiment scores are better at assessing general sentiment – like overall satisfaction or quality of interactions – than specific domains, like belonging or learning impact.
- AI NLP will blur the lines between quantitative and qualitative analyses. Quant and qual are analyses, not methodologies 🙂
