The human attention span is at an all-time low. Distracted by text messages and push notifications, the average person only manages to concentrate on a single task for eight seconds — a 33% decrease since 2000.
This is bad news for market researchers. If you rely on surveys to gauge consumer opinion, your data could be spoiled by responder dropout, straight-lining, and other biases related to our hyperactive brains.
Luckily, you can collect reliable insights by being proactive about data quality during questionnaire development. It’s time well spent to polish your survey upfront for accurate insights, rather than reactively manage quality concerns post-fielding.
First, it’s good to understand the dos and don’ts of effective survey design.
1. Don’t ignore mobile users. Do design screen-agnostic surveys.
Modern consumers are busy. They are constantly racing against a ticking clock, only pausing during fragmented bits of leisure called ‘time confetti.’ Most people spend these brief moments scrolling social media or answering emails.
As a market researcher, you must use screen-agnostic questionnaires to engage respondents during these brief moments. By optimising your surveys for mobile devices, you can reach 30 to 40% more survey takers and garner results that are more reflective of your target audience.
2. Don’t design long questionnaires. Do design 10-minute surveys.
As previously noted, the average attention span is waning. We are constantly faced with a barrage of phone calls, news articles, and enticing cat videos. Because of this, surveys should be kept short.
As a general rule, longer surveys correlate with higher instances of responder dropout. Research conducted by Kantar, for example, suggests that a survey over 25 minutes loses more than three times as many respondents as one under five minutes.
We like to aim for 10 to 12 minutes, but as always, consider your research and the overall panellist experience.
3. Don’t ask redundant questions. Do prioritize efficiency.
If you are struggling to trim a lengthy survey, start by looking for questions that prompt the same or similar responses. For example, if your questionnaire asks respondents to rank their favorite ice cream flavours in one question, there is no need for them to later select the most delicious flavours from a list.
Redundant questions can be confusing and frustrating for survey takers, triggering survey dropout. Market researchers can collect more valid and reliable data by eliminating all redundancies. You can do this intuitively by reviewing each question from the perspective of a respondent. Or, you can conduct a Pearson correlation using Excel.
4. Don’t use verbose language. Do keep questions clear and concise.
Nearly 20% of workers will stop reading an email after a paragraph. Keep this in mind when you are developing questionnaires. Most respondents are likely to abandon a survey when faced with wordy questions.
Or, they will select answers at random to speed things along.
To improve your questionnaire design, edit out unnecessary text.
Consider this verbose question: “Think of the last time you purchased an article of clothing (such as a blouse, jacket, sweater, or pair of pants). Your purchase may have been conducted online or in person. Now, please rate on a scale of 1-10 (where 1 is strongly pleased and 10 is strongly displeased) how satisfied you were with your purchase.”
Instead, you could simply ask, “Thinking about the last time you bought clothing, please rank how satisfied you were with your purchase.”
By paring down your language, you can garner more accurate responses. You can also reduce responder fatigue.
5. Don’t choose question types at random. Do think about the survey’s purpose.
Survey designers are often so fixated on question order and wording that they give little thought to question type. However, the types of questions you ask can significantly impact the survey experience and, therefore, data accuracy.
For example, you may use slider or grid questions to present multiple options — say, brand names — to a user at once. Or, if you need to present responders with mutually exclusive answer choices, you may opt for a multiple-choice question. Meanwhile, if you are hoping to garner qualitative insight from survey takers, you may opt for an open-ended question.
By thinking critically about the type of information you want to collect, you can select the most appropriate type of question. This, in return, improves data quality.
6. Don’t ignore survey takers’ needs. Do apply empathy to questionnaire design.
There is an undeniable science to designing effective and efficient surveys. However, you must also remember that the people answering your questions are, well, people. They are emotional and unpredictable beings who could potentially feel judged by the questions you ask.
Questions concerning salary, education status, or habits (e.g. ‘How often do you eat fast food?’) can feel interrogative and overly personal. As a result, respondents are liable to bend the truth, which leads to poor data quality.
Instead, consider adopting an empathetic approach to survey design. An empathetic approach is one that encourages you — the market researcher — to think from the perspective of a respondent. In doing so, you can help the survey taker feel more comfortable telling the truth.
In practice, empathetic survey design may involve reframing potentially loaded questions or using humor to diffuse a socially-charged prompt. For example, Kantar found that 27% percent of people who were shown a meme in the question “do you recycle?” admitted to never recycling. Comparatively, only 1% of respondents who weren’t shown the meme admitted to not recycling.
7. Don’t forget about quality assurance. Do conduct survey data quality checks.
Today’s survey respondents feel more time-pressed than ever before. Because of this, they are likely to rush through questionnaires or offer nearly identical responses to each question. This can skew the statistical accuracy of your survey.
As such, quality checks are considered a survey design best practice. Fortunately, these quality checks need not be complicated. You may, for instance, choose to incorporate a red herring into your questionnaire.
A red herring is a question that offers a bogus option among a set of valid ones. As an example, you may ask, ‘What is your favourite dessert?’ The answer choices may include ice cream, cake, pie, pudding, and furniture. Obviously, the latter isn’t a toothsome treat but rather a trick for catching respondents who aren’t paying attention.
Learn more or subscribe to monthly research tips
Well-crafted surveys are just one essential element for collecting honest and reliable marketing research data. Find our winning formula for quality data here.
For monthly survey design and sampling tips direct to your inbox, subscribe here. You'll also be notified when new Online Survey Training Modules are released.
Download the 12 tips for writing better survey questions
Fill out the form to download 12 tips that will help you write questions designed to:
- Maintain clarity
- Avoid bias or confused responses
- Be optimised on any device (like mobile!)
- Engage your audience
- Deliver reliable insights
