Combating Fraud in Market Research

A Conversation with Steven Snell on Now That’s Significant Podcast

In a recent episode of the Infotools podcast, Now That’s Significant, Michael Howard spoke with Rep Data’s Steven Snell, PhD, Head of Research, about tackling data quality issues in market research. Steven brings a wealth of experience in combating fraud and improving research strategies—and currently also serves as President of the Market Research Council, is a longtime member of the American Association for Public Opinion Research, and represents our company on the Advisory Board of the University of Georgia’s Market Research programs.

Fraud in Market Research: A Growing Concern

The conversation kicked off with Steven’s focus on diagnosing and fighting fraud in market research. He explains to Michael in the podcast that fraud has evolved significantly. While traditional methods such as attention checks and open-ended analyses were once effective, they are now struggling to catch sophisticated fraud, especially with the rise of AI technologies.

"The reason we’re so worried about fraud is that a lot of it doesn’t look like fraud," Steven shared. Fraudsters now use more sophisticated tools, creating responses that seem genuine on the surface. This technological advancement, paired with the surge of generative AI, has made it increasingly difficult for researchers to spot bad data. According to Steven, as much as 60-85% of fraud can slip through traditional data cleaning methods.

What Makes Fraud Hard to Spot?

During his conversation on the Infotools podcast, Steven highlighted the common types of fraud—bots, click farms, and hyperactive respondents—that can skew survey results. Fraudulent responses often have subtle signs: they don’t provide outlandish answers but tend to select fewer brands or make unremarkable choices. This can create a distorted middle ground in the data, blurring the lines between high and low-awareness brands. The result? Actionable insights become much harder to derive.

"Fraudulent responses attenuate findings and drag results toward the middle, making it tough to draw meaningful insights," Steven noted. This effect significantly impacts KPIs and decision-making, as responses from fraudsters tend to diminish the differentiation between strong and weak brands.

The Role of Paradata and Metadata

To combat this, Steven emphasized the increasing importance of paradata and metadata in fraud detection. Paradata and metadata refer to information about how survey responses are gathered, not the content itself. Metadata includes details like the device used, IP address, and browser, while paradata tracks behaviors such as time spent on a page or interactions with questions. These insights help identify fraud by revealing patterns in the data collection process that may not be apparent in the responses alone.

"Instead of just relying on response content, we should be looking at the metadata to identify if the person taking the survey is using a VPN, spoofing their IP address, or has attempted numerous surveys in a short period," he said. Tools like Rep Data’s Research Defender use these signals to block fraudsters before they even touch a survey, allowing researchers to ensure cleaner, more reliable data.

Empowering Researchers to Take Control of Data Quality

Steven shared on the podcast that his approach to improving data quality is rooted in empowering researchers. He pointed out to Michael that with the democratization of market research tools, the responsibility for data quality now lies with the researcher. "Everyone has an interest in data quality, but the researcher has the most at stake. If the data is fraudulent, it’s their insights that are at risk."

By using tools like Research Defender, researchers can configure thresholds for data quality and take charge of identifying fraudulent responses before they contaminate their datasets.

Navigating the Future of Market Research

Looking ahead, Steven sees a need for a more comprehensive approach to fraud prevention. As the technology behind fraud continues to evolve, so too must the strategies used to detect it. "Fraud prevention needs to happen before, during, and after the survey," Steven said. "We can’t just wait until the data is in. We need to be proactive."

He also raised concerns about synthetic data and the risks of training models on data sets that may already contain fraud. Synthetic data, while promising, can perpetuate errors if it’s based on flawed or fraudulent historical data.

Final Thoughts: A Call to Action

Steven closed the conversation on the Now That’s Significant session with a powerful reminder: "Not all bad data is fraud, but all fraudulent data is bad." Researchers need to look beyond just the content of responses and pay attention to the metadata, para-data, and behavior patterns of respondents. By doing so, they can ensure the integrity of their data and make more informed, accurate decisions.

For those interested in learning more about combating fraud in market research, Steven recommended checking out Rep Data’s blog, and our Fraudster of the Week posts on LinkedIn, where we share insights into the latest trends in fraud detection.