How to Deal with Bots in Market Research

by Patrick Stokes, Rep Data

Twitter is plagued by spam and fake accounts on the site – at least that’s the alleged reason that Elon Musk pulled out of the multi-billion dollar deal to buy the social media platform. The truth is that this kind of fraud is unavoidable, and I’d argue that it doesn’t take away too much from the overall value of the platform. After all, Twitter still has active users in the hundreds of millions, and many of them are quite influential in business and media circles.

In market research, we’ve been battling fraud for years. It’s pervasive, it’s persistent, and it certainly isn’t going to go away anytime soon – not when there is financial gain on the table for fraudsters taking surveys for incentives. Some sneaky characters that plague the industry are bots, presenting an automated threat to data quality. But we needn’t panic when we hear this word.

What do bots do?

Bots are not limited by any means to the market research space. They can attack websites to make them vulnerable to malicious activities, automate dishonest posts on social media (i.e. Twitter case in point), text your mobile phone, send out spam to create larger infections, and much more. The financial industry is the most affected, with a whopping 97% reporting in 2020 that an API had been attacked by bots. While the U.S. remains the epicenter of bot fraud, with a Globaldots report showing that it represents 53% of bot traffic, it is an international issue.

Related

Applying the Swiss Cheese Model to Survey Fraud

Not all bots are bad – think about Googlebot’s search automation function – but the ones we are fighting in survey research certainly are. And the game of fighting them has become a moving target, as they become more and more sophisticated and complex. Our defense against them must undergo continual optimization and improvement. We need a wide mix of methods to stop bots by quickly identifying them and removing them from surveys before they affect our data quality.

Mitigating market research fraud

The first step is employing the right approach to fraud mitigation. We won’t ever get rid of these little guys completely, but reducing their impact is critical when it comes to data quality – the central value proposition of market research itself. There are a few best practices that we’ve seen work when it comes to selecting sample to help reduce bots and fraud.

• Unbiased sample sourcing: Today’s panel or sample companies all have some level of fraud detection in place by necessity. These approaches have varying levels of success. By not owning a proprietary panel, and sourcing respondents from multiple panels and sources, we’ve found that we can essentially place another set of protective layers between a survey and potential fraud. The panel companies from which we source sample have their own mitigation techniques in place, and then we place our own fraud fighting machine on top of that. In addition, multi-sourcing every study with multiple panels spreads out the risk of a bot infiltrating a survey. It works.
• Multiple techniques: Layering fraud mitigation techniques is proven to help boost data quality – we did some research-on-research on this topic a couple of years ago. By employing a number of fraud mitigation measures, you can start to proactively eliminate fraudulent respondents/bots. There are a bunch of techniques out there such as: cross-checking respondents for blacklisting in external databases; tracking the rate of respondent activity across the market research ecosystem; text analysis via open-ended questions; verifying emails and IPs; and more. Beyond some more common approaches, sample providers are getting creative to fight fraud. For example, our proprietary hidden variable technique is specifically designed to trip up bots, as it is only visible to them and not humans.
• Continuous improvement: Bots aren’t resting on their laurels after they infiltrate a few surveys. In fact, they’re smart enough to evolve and circumvent mitigation techniques on a regular basis. This means we have to be vigilant in continually updating our own techniques to identify and fight them. As fraud becomes more sophisticated and complex, and bots mimic human behavior more and more accurately, we must be ready with the necessary tools to identify them and remove them from survey projects.

The truth is that there may always be bots in panels, no matter what protocols are in place. There is money on the table, in the form of incentives, and that is a strong motivator for bots to infiltrate surveys. Of course, we should, and must, continue to fight the good fight to keep them out and reach our data quality goals. But we must also keep calm, carry on, and realize that the cat and mouse game of chasing bots is part of the market research ecosystem. We just have to be better at finding them than they are at showing up.

Posted on: Greenbook Research Methodologies