There are various goals to an open-ended question. In most cases, however, goals are not about the volume of feedback – they are about the quality. Whether you are trying to get respondents to be analytical, creative, or spontaneous, the biggest challenge you face is encouraging people to think and think in the right sort of ways.
Most respondents spend 15 seconds answering the average open-ended question and provide an average of five words. Five words might be enough if it constitutes meaningful feedback, but often it’s humdrum verbiage, which is a bit of a nightmare to analyse.
If you were to ask people to watch an ad and write down what they thought of it, the most common response would be "it was OK." The second- and third-most common remarks would be, "I liked it" and "I didn't like it," and the fourth would be "I don't know." These responses are clearly not a great deal of value, so you might as well have asked a yes/no question, “did you like it or not.”
Here are 9 of our best practices for eliciting more thoughtful feedback with open-ended questions. Find the complete guide, with all 23 tips, here.
- Bring respondents into the problem
- Challenge respondents
- Use word counters
- Deal with spelling and literacy concerns
- Tackle boredom thresholds
- Ask a seed question
- Set rules
- Make it personal
- Realize the power of the word "imagine"
1. Bring respondents into the problem
Bring respondents into the problem you are trying to solve if possible. Instead of asking "Why do you like this brand?" you might say, "This brand is a lot more popular amongst some people than others and we are trying to understand why."
2. Challenge respondents
Using language that challenges respondents to do something can be an extremely powerful weapon to encourage feedback. Instead of asking "What brands come to mind?" you might re-phrase that to say, “How many brands can you guess?" Or, instead of saying "What words come to mind when you think of this brand," you might say, "can you think of the most popular words people associate with this brand?"
Kantar has discovered that this type of approach can sometimes double the level of feedback as well.
3. Use word counters
Asking people to write not more than “X” number of words on a topic and using a word counter so they can see how much they have written is a great way to extend the volume of open-ended feedback. The “X” limit might be 3 words or 300, depending on the circumstances.
4. Deal with spelling and literacy concerns
One of the reasons people can be reluctant to give open-ended feedback in surveys is because they are worried about their spelling and how well they can write. There are a couple of ways of dealing with this. First, make it clear that you are not worried about it. Secondly, we have found that showing a range of informal feedback, with not necessarily perfect spelling, encourages respondents to be less concerned about their personal style of delivering feedback.
5. Tackle boredom thresholds
People put far less effort into answering open-ended questions at the end of a survey. There are often trigger points you will notice wherein people tend to give up, and their desire to get to the end of a survey cuts in so strongly that their responses to open-ended questions dry up.
6. Ask a seed question
One of the best tricks to get people to put more thought into answering an open-ended question is to ask a question first that seeds a thinking process. For example, asking “Do you think your bank is ‘perfect?” This then seeds a question about what a bank could do to improve its service.
7. Set rules
Adding a rule to a question can make an open-ended question a whole lot more interesting and fun for respondents to answer, thus triggering more thoughtful feedback. The easiest way to do this is by placing word limits - e.g., “review this film in 3 words.”
8. Make it personal
Always remember you are one human being talking to another, and most people's favourite topic is themselves. Try, wherever possible, to frame the question in a personal context. Instead of asking “Why do you like this brand?” you might ask, "If a friend asked you what you thought of this brand, what would you say?"
9. Realise the power of the word "imagine"
The word “imagine” is a magical word to use as a means of getting people to think. Kantar Profiles did an experiment in which we asked one group of people to make a list of their favourite shops. We asked another group, “Imagine you could design your perfect shopping centre. What shops would you have in it?" This increased the feedback 4-fold.
What can you achieve if you get it all right?
In experiments where we redesign the way we ask an open-ended question, the biggest uplift in feedback we have ever achieved has been a 6-fold increase from 17 words to 103 worlds.
This used a combination of techniques listed above. With the right thought and approach, you can easily achieve double or treble the volume of feedback by re-engineering the wording of a question.
But note that, as we said at the start, it is often not about volume of words so much as about the clarity of thinking these words stimulate. We find that with increases in volume feedback, is also linked to more thoughtful feedback.
What do you do with all that open-ended data?
All this begs the question about what you do with all this open-ended feedback. And that can be overwhelming. Some clever ways of automatically processing open ended feedback have emerged with social media text analytic software and natural language computational techniques. You can also try Kantar Profiles’ Taxonomy Builder, a free excel tool, here.
Access all 23 tips
To read all 23 tips for collecting more thoughtful responses from open-ended questions in online surveys, use the form below to download the complete guide. For more on this topic, watch the on-demand webinar with Jon Puleston.
