How to Evaluate Customer Survey Responses
There are many ways to measure customer feedback. The Customer Satisfaction Score, Net Promoter Score, and Customer Effort Score are just some of the available metrics you can compute to gauge customer feedback. The ways to measure feedback depends primarily on whether you are working with open-ended or closed-ended questions.
Closed-ended question methods
Top 2 Box score
The Top 2 Box score is a common way to compute metrics like the Customer Satisfaction Score and Customer Effort Score. It is calculated by taking the percentage of respondents who gave a score within the top two available choices.
For example, taking the percentage of respondents who answered “9” or “10” on a 0-10 scale question would give you the Top 2 Box score for a traditional question like, “On a scale of 0-10, how satisfied are you with this product?”
Average and Median score
The easiest way to measure customer feedback is to simply take the average or median score. The process is simple and interpreting the results is straightforward.
However, using only the average or median in your analysis can produce some misleading results. The simple mean or median doesn’t provide any information on the distribution of responses. An average result of 5 could mean that most respondents gave a rating of around 4-6, but it could also mean that there are two equally-sized clusters around the top (9-10) and bottom (1-2) of the scale.
The mode is simply the most-picked answer. The modal outcome is particularly useful when analyzing responses from multiple-choice questions. When you only need to know what the most popular response is, the mode will be your go to. However, we recommend also considering the distribution of your responses, along with the mode. You may find that there are multiple answers with unusually high scores.
We have created a customer satisfaction tutorial that shows how easy it is to create a Top 2 Box Score and calculate the average and median.
Net Promoter Score
The Net Promoter Score (NPS) has become a staple of many customer feedback surveys. it's a great way to measure customer loyalty and predict customer retention rates.
Then you calculate NPS with the following formula:
NPS = Percentage of Promoters – Percentage of Detractors
- Detractors: respondents who gave a score between 0-6
- Passives: respondents who gave a score between 7-8
- Promoters: respondents who gave a score between 9-10
Open-ended question methods
When it comes to open-ended responses, there’s no substitute for human judgment. Data algorithms are prone to misinterpreting and mischaracterizing customer feedback when written in plain-spoken language. No matter how advanced an algorithm is, it can't understand your customers as well as you can.
Even if you plan to use a wide array of machine learning and text processing algorithms, it’s always beneficial to also have a person analyze the responses.
Sentiment analysis algorithms can take comments from your respondents and measure how positive or negative the feedback is. It assigns a score to individual words and phrases based on their definitions and the context in which they appear. For example, the word “terrible” has a negative score while a word like “wonderful” has a positive one.
This technique can be useful, but you should be aware of its many shortcomings. Sentiment analysis algorithms are notoriously bad at detecting sarcasm and can mislabel colloquialisms, idioms, and unusual phrases.
Topic modeling algorithms detect common keywords and phrases in your customer feedback data. For example, the words “lag”, “slow”, and “loading time” could be grouped together to identify respondents who complained about the speed of your program. People who used positive words ("good", "great", and "helpful") alongside words like "support" could be considered customers who are satisfied with your customer service.
Like with sentiment analysis, topic modeling algorithms can mischaracterize your feedback and skim over issues that a person would definitely pick up on.