Survey data is a vital part of the information used in market research for informing decisions on users, customers, and products. Unfortunately, however, dishonest people will try to cheat the system to earn rewards, either by offering disingenuous information or intentionally hacking surveys.
In the wake of the pandemic, this fraud is becoming increasingly prevalent. The good news? Kantar’s Profiles division has been extensively researching fraudulent activity and has developed Qubed: an advanced machine learning and AI system designed to detect signs of fraud and poor user behaviour quickly and act accordingly. By using a wide range of quantifiable measures and behavioural analysis, Qubed can detect problems faster, more accurately and tackle problems previously ignored by traditional survey quality systems.
Below, we offer more insight into the dangers of survey fraud, how Qubed safeguards data from fake surveys, as well as specific actions researchers can take to support quality data returns.
What is survey fraud?
Brands funnel billions of dollars into market research each year with hopes of better understanding consumers and, in doing so, making informed business decisions. Nevertheless, data yielded from this costly and time-intensive research can be spoiled by survey fraud.
In simple terms, survey fraud occurs when panellists offer disingenuous responses. This fraudulence comes in many different forms, some more innocent and others quite malicious. For example, a busy respondent may straightline their way through a lengthy boring designed survey they aren’t fully engaged in. Meanwhile, some people make their livelihood by hacking surveys and receiving rewards in bulk. Though these two behaviours are quite different, both types of online survey scam can tank the validity and reliability of your data.
Worse yet, survey fraud is a growing problem. In Q4 2022, Kantar found that researchers are discarding up to 38% of the data they collect because of quality concerns and panel fraud. That means more than a third of the money your company invests in survey sampling could be blatantly wasted – or worse yet – cause you to make poor business decisions.
Fortunately, there’s a better solution. With Qubed, our proprietary anti-fraud technology, we ensure our panellists are real humans engaging with your study—not bots, not hackers, or groups of out of country users – who are intent on wasting your money and time. In return, you can expect a dramatic reduction in fraudulence and an invaluable increase in data quality.
The different types of survey fraud
There are three different types of survey fraudsters. Though each has unique motivations, all have the potential to skew survey data.
The Lazy Panellist
The Lazy Panellist is characterised by a lack of diligence. This individual is a multitasker who isn’t interested in investing sufficient time or energy into completing a survey. As a result, they may:
- Straightline answers
- Speed through questions
- Skip questions altogether
- Provide insufficient answers
- Enter gibberish into open-ended fields
Lazy behaviour can sour your data. However, this type of panellist shouldn’t be disregarded entirely. There are many different factors that could have contributed to their behaviour. For example, they may have started the survey at an inconvenient time. Or, the survey itself may have issues that contributed to the response biases.
The Dishonest Panellist
The Dishonest Panellist commits identity fraud in an attempt to qualify for as many surveys as possible. For example, this panellist may lie about basic information like income or occupation. Or, they may even go as far as to create fake profiles.
Since honesty is a concern, this panellist should be blocked from all surveys.
The Fraudulent Panellist
The Fraudulent Panellist is working alone or in a group to intentionally hack surveys. Often operating overseas, these imposters are motivated by monetary rewards and aim to collect these rewards as quickly as possible. Unfortunately, these fraudsters are very sophisticated and difficult to identify.
Market researchers should make every effort to thwart fraudulent panellists from accessing surveys. Nevertheless, this can be challenging without artificial intelligence or a machine learning tool.
How to identify survey fraud
Fraudulent panellists are becoming increasingly savvy. Intent on collecting big sums of money and disguised under multiple IPs, ISPs, Devices and personas, they manipulate the survey experience by either giving false data or generating Ghost Completes (completely by-passing your study). Because of this, it can be difficult—if not impossible—to detect survey fraud. Until Qubed.
Qubed uses state-of-the-art artificial intelligence to process more features and identify fraud in real-time where humans cannot. More specifically, Qubed works by:
- Assessing Domain Knowledge: Qubed is the only software on the market that has vast amounts of domain knowledge on survey fraud, from the most recent angle the frausters are exploiting to the high risk IPs the hackers have used in 2018. Qubed is ahead of the frausters and constantly learning from them. Reviewing Key Metrics: Qubed draws conclusion on users based off a multitude of metrics, from IP, ISP, device fingerprint, to demographic sensibility, demographic consistency, hour of activity consistency, survey taking behaviour, Open-ended response test etc. All of these metrics and features are evaluated by a centralised AI to make objective decisions on survey fraud.
- Evolving: Qubed is not a static software. Through real-time machine learning, it is constantly evolving to outsmart hackers and improve data quality. Qubed learns new emerging patterns, breached ISPs and group hacking activity as they happen to minimize damage done and to block the perpetrator real-time.
- Pinpointing Fraudulence by Type: Not all quality concerns are equal. Straightlining, for example, is very different from bot activity or ghost-completes. Fortunately, Qubed is advanced enough to not only identify survey fraud but to also identify the specific type of survey fraud. This allows different handling of the users, where some diligence problem could be solved through better survey designs or communication/education with the user.
In short, Qubed is the most intuitive tool in your arsenal against survey fraud. Backed by four years of extensive research, this flagship product can detect and remove even the most sophisticated of scammers.
Tips to prevent survey fraud
As long as there is an incentive, there will always be fraud attempts by the fraudulent panellist. These respondents are aggressive about extracting economic value and often operate in organised groups. They cannot be detected using simple measures like IP address or device type because they know to hide these signals. Ergo, the only way to combat these scammers is with advanced systems like Qubed.
By comparison, companies can take preventative steps to address quality issues associated with lazy and dishonest panellists. More specifically, market researchers can ensure that they follow survey design best practices.
These tenets increase overall research effectiveness by reducing leading language, conformity bias, acquiescence bias, and other non-sampling errors that are souring the quality of your data. Ergo, as a general rule, market researchers should make every effort to:
Remove ambiguity
Vague and cryptic questions can be frustrating, even for the most motivated of survey participants. Because of this, questionnaires should use clear, concise language that’s free of industry-specific jargon and acronyms. You may also consider incorporating instructional text to help guide participants through the survey.
Take an empathetic approach
When you’re creating a survey, it’s easy to become overly fixated on the research goal and forget that actual humans will be completing it. That’s why you must adopt an empathetic approach to survey design. An empathetic approach essentially requires that you view the survey from the respondent’s perspective. You may ask yourself questions like:
- ‘Did I become bored while taking the survey?’
- ‘Did certain demographic or behavioural questions make me feel judged?’ (For example, asking if the survey-taker completed college may make non-graduates feel inadequate.)
- ‘Does the survey feel interrogative or overly invasive?’
- ‘How long did the survey take to complete?’
If you’re having trouble separating yourself from the survey, consider having colleagues test the questionnaire. They may be able to offer a more honest assessment.
Ask questions that can be answered
This suggestion may seem obvious enough. However, too often market researchers present survey respondents with questions that they don’t have enough knowledge to answer. For example, a questionnaire may ask participants about their preferred investment instrument when they know nothing about investing. This confusion will either cause the panellist to lie or abandon the survey altogether—a phenomenon known as survey dropout.
Consider survey length and complexity
Alas, confusion isn’t the only cause of survey dropout. According to research conducted by Kantar, surveys that take more than 25 minutes to complete lose more than three times as many respondents as those that can be completed in under five minutes. That means you should intentionally design questionnaires to be short, sweet, and simple. If you’re having trouble with this, look for redundancy or overly complicated questions. You may even reconsider the purpose of your survey.
Combat survey fraud and boost survey quality with Kantar
Ensuring data quality is essential in securing confidence in research conclusions. As more and more data gathering has moved online, the analysis of quality has become focused on questioning respondent identity and online engagement. But alas, scammers have developed savvy techniques to dodge these quality checks. That’s where Kantar's Profiles division comes in.
As a leader in market research, we safeguard the validity and reliability of your hard-earned data using Qubed. A proprietary anti-fraud technology informed by years of extensive research, this software quickly and intuitively identifies fraudulent scammers using machine learning. In return, Kantar’s Profiles division is seeing an average of 6% of data rejected for quality where Qubed is running—a dramatic reduction from the industry average of 38%.
In addition to Qubed, we utilize more than 10 proprietary tools and techniques that are designed to combat survey fraud. For example, our technology employs methods that prevent multiple surveys from being submitted from the same IP address. This thwarts overseas scammers from working in unison to extract economic incentives. To reduce the possibility of bots, we also require that survey participants validate their email addresses.
In sum, if you want to make informed business decisions without the threat of survey fraud, you must partner with Kantar.
