Common Mistakes of Survey Design
When it comes to writing survey questions, there are many landmines to avoid. A poorly written question can decrease your response rates, muddy your data, and frustrate your customers. Here are six common mistakes to avoid.
Loaded and leading questions
When writing questions for your customer feedback survey, you want respondents to be able to answer as freely and honestly as possible. This means avoiding loaded and leading questions. A loaded question is one with an in-built assumption, and a leading question is one that nudges the respondent to answer in a particular way. Here are some examples:
- Loaded question: Where is your favorite place to drink alcohol?
- Leading question: How would you rate our exceptional customer service?
The loaded question assumes that the respondent drinks alcohol, and the leading question assumes that the respondent agrees that your customer service is "exceptional." A well-written question would strip away any unnecessary assumptions.
Related questions should always be grouped and asked in sequence, using a technique called "laddering." A respondent can be asked how satisfied they are with a particular feature or product, and in the next question be asked to elaborate. This allows respondents to focus on one thing at a time. If these questions were scattered throughout the survey, the respondent might feel overwhelmed and forget how they answered the previous questions.
Incomplete and mutually non-exclusive response categories
Design multiple-choice questions so that the response categories are complete and mutually exclusive. In order to be complete, the response categories must cover all possible answers. This often involves including a "None of the above" or "All of the above" category. For the response categories to be mutually exclusive, there must only be one correct answer.
Unintentionally vague questions
Sometimes you want to include a question that can be a little vague. Something like, "What else would you like us to know?" is a popular way to conclude a survey. However, unintentionally vague questions can be difficult to analyze. There is a world of difference between "What do you think about this product?" and "What do you like about this product?" Be sure of what you're actually asking!
Each question should ask one thing and one thing only. A question like, "What did you think about the price and quality of this product?" is what is known as a double-barreled question. It asks for the customer's opinion on two different things: price and quality. The customer's response could be in regards to the price, the quality, or both. You won't necessarily know which. Instead, ask two separate questions or pick the attribute you most want to focus on.
Too many questions
We're going to keep coming back to this one. If a question is not essential, omit it! Consider the time commitment you are asking from your customers, and give some serious thought to how many questions your customers are willing to answer. As a general rule, you should avoid designing a survey with over ten questions. That's usually a surefire way to lower your response rates.