6 Ways to Improve the Data Quality of Online Quantitative Surveys

1. Design surveys carefully

High quality data collection starts with a well-written questionnaire that focuses on the survey objectives. This sounds obvious, but you'd be surprised! Make sure you write concise questions that capture key information using the vocabulary of your targeted respondent. This will make it clearer and easier for them to understand what you are asking. One way to assess the questionnaire readability and to uncover any key missing aspects is to share it with a friend or co-worker for feedback.

2. Screen survey participants

Ask questions that qualify respondents for the survey and terminate those that fail the screening criteria. Avoid designing questions that provide clues about what you are looking for and therefore encourage participants to select answers they believe will qualify them for the survey.

3. Allow honest answers

Including "don’t know" and "none of these" in answer lists improves data quality. Not having these options to select from when a respondent doesn’t know, or can think only of an answer that is not provided, forces them to select an inaccurate answer even though they had no intention of giving false information.

4. Implement quality control checks

 You can separate the good from the bad respondents by implementing quality control checks in the survey. As a rule of thumb, only discard respondents that fail 2 or more quality checks.

• Attention checking: Add a single choice question somewhere in the questionnaire asking them to select a specific answer:

• Front/back validation: Ask the same question twice. For example, ask them to type their age and in another question ask to select their age bracket. The first answer should be within the selected age bracket.
• Speeding: Assuming all respondents see all questions, it is an industry standard to remove those who complete the survey in less than 1/3 of the median survey length.
• Flatlining: When asked to rate statements or brands in a grid question, some participants might legitimately select the same answer for all, forming a perfect flatline or flatline due to fatigue or boredom. This is what a flatline looks like:

 

Removing flatliners may not be appropriate depending on the topic or content of the question. Add contradicting statements to catch flatliners and if not possible, analyze flatlining on a question by question basis in combination with other quality control checks.

• Open-ended gibberish: Make open-ended questions mandatory and if the answers provided are illogical, review all answers from the respondent to decide if it should be removed or not.
• Red herrings (fake brands): Catch inattentive respondents by including fake brands in an answer list. The fake brands shouldn’t have similar spelling to existing brands or be an existing brand in another place. Flag respondents that select 2 or more fake brands.

5. Optimize for mobile

Every respondent has a preference regarding choice of device to complete surveys. Optimize your survey for all device types: mobile, tablet and PC. When respondents need to zoom in and out to answer questions due to poor rendering, it can really impact your data quality.

6. Ensure accurate survey programming

When a questionnaire has logic applied, meaning, a question must be asked if certain answers were selected before, make sure you test the survey logic before going live. Use testing data to spot check issues with logic and do a thorough check after soft launching the survey. Only proceed with full launch when you have achieved 100% certainty of survey programming.

Check out more market research tips here!