Surveying is a powerful tool in your market research toolbox. The resulting data can give you a better understanding of consumers, allowing your company to make informed business decisions. But alas, no one said conducting an effective online survey was easy. Below, we delve into the best practices of questionnaire design while presenting turnkey solutions that simplify and streamline the survey design process.
How to conduct a survey
Before delving into the how-tos, it’s helpful to start with the basics. What are online sample surveys?
Online sample surveys are questionnaires that are sent to a sample of individuals from your target population (aka target audience). An online survey offers invaluable insight into consumer opinions, preferences, and purchasing behaviors. However, conducting a questionnaire that yields valid and reliable data requires a meticulous approach.
Before you can begin firing off questions, you must determine your intentions and target audience. What do you want to learn more about and from whom? A company may, for example, want to assess laundry detergent brand awareness among millennial consumers.
After establishing this baseline, you must write the questions. Though this sounds simple enough, crafting effective survey questions is equal parts art and science. These prompts must be clear and concise to avoid survey dropout, straightlining, and other practices that contribute to dubious data. However, the questions must also offer the depth needed to make meaningful observations.
Additionally, market researchers must select the appropriate question type for the situation. Misusing a Likert scale, for instance, could result in tainted data. Meanwhile, overusing open-ended questions could tempt participants to enter gibberish that’s not in the least bit helpful.
What makes a good survey question?
Thus far, we have alluded to what constitutes a bad survey question: bias, verbose language, repetition, and so on. But what makes a good survey question?
1. Mobile-friendly
Consumers crave a fast, frictionless experience. That means survey questions must, above all, be screen agnostic. All the answer choices should fit on a mobile screen, for example.
2. Short and succinct
Attention spans are waning. As a market researcher, you often have less than 10 minutes to engage survey participants. If your questions are too long or too confusing, consumers might drop out or give unreliable answers by straightlining, speeding, or random question answering.
3. Fun and engaging
The more tedious your survey, the less reliable your data. It’s that simple. So, your goal as a market researcher is to make your questionnaire as fun and engaging as possible. Can you incorporate iconography? Casual language? Meaningful questions consumers actually want to answer?
4. Unbiased
If you want to collect data that will help your company make informed decisions, survey questions must be free and clear of leading questions, loaded questions, double-barreled questions, and other types of bias.
5. Meaningful
Survey dropout is sure to skyrocket if you ask the same question or a similarly-phrased question repetitively. Ergo, make sure to ask meaningful questions once and only once.
Types of survey questions
The significance of selecting the right type of survey question is often overlooked. This is unfortunate because choosing an unsuitable question type can contribute to response biases that spoil your data. The corollary, of course, is that you can optimize your survey data by opting for questions that suit the framework of your survey.
Generally speaking, there are five different types of survey questions:
1. Single and multiple choice questions
Single and multiple choice questions are examples of close-ended questions. Meaning, there are a finite number of responses. The difference between the two is that with single choice questions, participants can only select one option. Comparatively, multiple choice questions are characterized by numerous options.
Single and multiple choice questions are among the most commonly used types of questions because they are quick and simple to answer. However, one downside is that the data yielded is purely quantitative.
Pros
- Quick and simple to answer
- Mobile-friendly
- Easy to analyze
Cons
- Time-consuming to generate
- Yield purely quantitative data
- Limited responses
2. Open-ended questions
Open-ended questions are prompts that cannot be answered with a simple ‘yes’ or ‘no.’ These questions are explanatory in nature and yield rich qualitative data by way of free-form text responses. This qualitative information can be used as is or transmuted into quantitative data via word clouds and other tools.
Open-ended questions do have a downside, however. Namely, they are time-consuming to answer. Because of this, if your question aren’t engaging, and you use too many, you might not collect the quality you can expect.
Pros
- Unlimited responses
- Richer, deeper data
- Offers insight into the customer journey
Cons
- Time-consuming to answer
- Yield unusable responses (i.e., emojis, gibberish, URLs)
- Difficult to analye without the proper tools
3. Rating scale questions
Rating scale questions present participants with a spectrum of answer choices pertaining to a specific theme. A popular example is a rating scale that assesses a consumer’s likelihood to purchase a product again, with 1 representing ‘very unlikely’ and 5 representing ‘very likely.’ Frequency is also commonly assessed with rating scale questions.
Rating scales are helpful because they allow market researchers to quantify subjective sentiments. The disadvantage is that rating scales are inherently subjective. Different participants often have different interpretations of vague verbiage like ‘sometimes’ or ‘very likely.’
Pros
- Quantify subjective sentiments
- Easy to understand
- Affords simple analysis
Cons
- Answer options are vague
- Error of central tendency (participants tend to select the midpoints of the scale)
- Limited options
4. Likert Scale questions
Likert Scale questions are those that employ a five- to seven-point scale in an attempt to quantify subjective sentiments. These questions are very similar to rating scale questions, but with a few important differences. Namely, a rating scale can consist of any number of choices while Likert Scale questions tend to offer either five or seven options.
Likert Scale questions are quite popular because they afford more granularity than binary questions. However, since market researchers generate the responses, these questions can be susceptible to leading bias.
Another issue is that respondents can find Likert Scale questions repetitive and fatiguing, leading them to lose concentration and not carefully consider their answers. Try to keep banks of such questions to ten iterations or less if possible.
Another problem with Likert Scale questions is mobile responsiveness. Oftentimes, seven answer choices cannot fit onto a small smartphone screen. As a result, the text is cut off, completely skewing the data yielded from the survey. Ergo, if you’re going to use Likert Scale questions, it’s a best practice to opt for five answer choices.
Pros
- Quick to complete
- Very popular question type
- Effective tool for gauging satisfaction
Cons
- Susceptible to leading bias
- Subjective answer choices
- Limited responses
- Repetitive
5. Matrix/grid questions
Matrix/grid questions group several multiple choice questions together in a grid of rows and columns. The first row presents the various questions while the column headers present the answer choices.
Since matrix/grid questions group multiple questions together, it gives survey takers the illusion of a single question. This offers the benefit of saving space and reducing survey drop-off. (Respondents would rather answer a single question than five different questions.)
However, it’s critical to understand when and when not to use matrix/grid questions. Why? Because these questions are often incompatible with smart devices. Because of this, surveys with matrices/grids often have a high dropout rate.
5 tips for designing a survey
If you want to design a survey that yields valid and reliable data, consider these five tips.
1. Determine your intentions
Before you conduct a sample survey, you must determine what information you hope to ascertain. Do you want to gauge consumers’ brand loyalty? Or are you conducting ad testing? Your intentions will affect the type of questions you ask.
2. Identify your target population
Next, you must identify your target population. Your target population consists of the individuals or businesses to whom you hope to sell products or services. For example, the target population for a software developer that sells small business software would be small business owners and finance professionals.
Since culling a group of willing participants can be quite time-consuming and expensive, many companies collaborate with market research partners. Kantar Profiles, for example, offers access to an exclusive network of 169 million research-ready respondents who have passed quality checks to eliminate fraudulence.
3. Choose the appropriate question type
As previously noted, choosing the right type of question can mean the difference between valid and invalid data. If, for example, you overuse open-ended questions, you could end up with a glut of gibberish answers. Comparatively, if you exclusively employ single choice questions, your survey probably won’t offer much insight into your target audience.
4. Consider the design elements in your survey
How your survey looks matters just as much as its content. Namely, if you don’t design a survey for mobile, you will see high rates of survey dropout. The responses you do collect won’t be representative of your target population.
Beyond mobile usability, the aesthetics of your survey can make or break engagement. Consider this as an example: One survey uses a small, illegible font and no iconography. Another employs large, easy-to-read writing and graphics. Which survey would you rather take?
5. Test your survey
Before you send your survey out to consumers, it’s important that you test it. This step need not be complicated. You can simply ask two to three colleagues to review the form. Afterward, ask for feedback. How long did it take for them to complete the questionnaire? Did all of the questions make sense? Was the format easy to follow?
Hearing from people you know can be a good reminder that actual human beings will be completing your survey—a realization that underpins empathetic survey design.
Survey samples
When optimizing a survey, it can be helpful to see sample survey questions. These examples can act as a guide, providing insight into how to structure different types of survey questions.
Single and multiple choice questions
Example A
Out of the options below, which automobile manufacturer do you consider the most reliable?
- Toyota
- Volkswagen
- Ford
- Mercedes-Benz
Example B
Which of the household cleaning product brands do you use on a daily basis?
- Windex
- Clorox
- Mrs. Meyer’s
- Lysol
Open-ended questions
Example A
How does Product X make you feel?
Example B
What steps do you take to research a product before buying it?
Rating scale questions
Example A
How likely are you to recommend Product X to a friend on a scale of 1-5?
Example B
Likert scale questions
Example A
How strongly do you agree with the following statement: ‘The payment process at Company X is simple and easy.’
- Strongly disagree
- Somewhat disagree
- Neither agree nor disagree
- Somewhat agree
- Strongly agree
Example B
How satisfied were you with your customer service?
- Very dissatisfied
- Somewhat dissatisfied
- Neither dissatisfied nor satisfied
- Somewhat satisfied
- Very satisfied
Example C
Matrix/grid questions
Example A
Example B
Explore the full training video on questionnaire development
A lot of thought goes into crafting a questionnaire. It may need to be an iterative process: from getting at critical questions, to following best practices, to providing a smooth experience for respondents, to considerations for data quality. All together, these actions increase respondent engagement, reduces bias, and encourage honesty.
Watch our 14-minute Online Survey Training Module: How to Create a Questionnaire. There you’ll learn more about the four steps for questionnaire success and additional considerations for creating a survey. Find our full library of video modules here.
Subscribe to learn more
Interested in learning more about designing effective online surveys? Fill out the form below to receive a monthly research tip on online survey design, sampling, and connecting data. You'll also be notified when new Online Survey Training Modules are released.
