We ran a Christmas survey with 1,000 Gen Pop completes, screened for those who celebrate Christmas, and asked them a single open-end…

The question: “In your own words, what is your favorite part of Christmas?”

Each response below is broken out by Real vs. Fraud as identified by Defender, using our three diagnostic measures:

1. Technology Threat Score (known & emerging fraud tech identification)
2. Survey Hyper-Activity Score (unrealistic number of surveys completed per 24 hours)
3. In-Survey Behavior Score (sophisticated bot or inattentive human behavior)

Below are the 16 open-ends broken into 5 comparison groups we wanted to highlight. These illustrate just how convincingly fraudulent respondents can mimic genuine participants. You'll notice:

• Nearly identical phrasings
• Punctuation & spelling misses
• Human-like emotional language
• Realistic narrative structure
• Seasonally appropriate detail

Even identified fraud users can generate open-ends that look perfectly valid at first glance.

Example 1

Family togetherness is one of the most common themes in Christmas open-ends. In theory, this should make fraud easier to spot… but in practice? Not anymore. As you’ll see in this example, spelling and punctuation aren’t the giveaways they used to be.

Real Respondents

• “Being worth my family”
  • Technology Threat: 21
  • Survey Activity Threat: 2
  • In-Survey Threat: 7
• “Being with me family”
  • Technology Threat: 12
  • Survey Activity Threat: 4
  • In-Survey Threat: 8

Fraud Respondents

• “Being with family.”
  • Technology Threat: 20
  • Survey Activity Threat: 36
  • In-Survey Threat: 10
• “Being with my family”
  • Technology Threat: 2
  • Survey Activity Threat: 72
  • In-Survey Threat: 0

Ironically, the fraud users were the ones with cleaner spelling, the opposite of what many researchers may still assume, but Defender’s activity signals lit up immediately. Gotta wonder if they’ll also attempt that many surveys on Christmas, too.

Example 2

“Spending time with family” showed up across many of the responses, real and fraudulent. That commonality makes it nearly impossible to rely on content alone. And in this case, even small narrative variations didn’t correlate to fraud patterns the way intuition might suggest.

Real Respondents

1. “My favorite part is spending time with friends and family.”
   1. Technology Threat: 3
   2. Survey Activity Threat: 5
   3. In-Survey Threat: 0
2. “Spending time with family.”
   1. Technology Threat: 1
   2. Survey Activity Threat: 2
   3. In-Survey Threat: 8

Fraud Respondents

1. “my favorite part is spending time with my family”
   1. Technology Threat: 28
   2. Survey Activity Threat: 31
   3. In-Survey Threat: 1
2. “Spending time with family and friends”
   1. Technology Threat: 69
   2. Survey Activity Threat: 3
   3. In-Survey Threat: 0

If we had relied on narrative cues alone (like mentioning friends), we would’ve thrown out perfectly valid completes. Instead, the threat profiles told the real story.

Example 3

Gift-giving is another classic Christmas trope. Heartfelt, universal, and unfortunately very easy for AI-driven fraud to imitate.

Real Respondents

1. “Giving gifts to my loved ones”
   1. Technology Threat: 20
   2. Survey Activity Threat: 1
   3. In-Survey Threat: 0
2. “Giving probably”
   1. Technology Threat: 3
   2. Survey Activity Threat: 2
   3. In-Survey Threat: 12

Fraud Respondents

1. “Giving gifts to my wife and children”
   1. Technology Threat: 3
   2. Survey Activity Threat: 37
   3. In-Survey Threat: 6
2. “Giving presents”
   1. Technology Threat: 88
   2. Survey Activity Threat: 7
   3. In-Survey Threat: 20

All four respondents expressed the same warm sentiment, but only two of them were legitimate. One fraudster leaned heavily on sophisticated fraud tech. Another questionable respondent simply attempted 37 survey in 24 hours, do you want that respondent in your survey data? The open end content didn’t differentiate them at all, but their behavior & technology did.

Example 4

Big Christmas dinners and gift exchanges are common themes in holiday surveys, nothing unusual there. But, when a respondent gives a perfectly reasonable answer and logs an astronomical number of surveys in a single day, the open-end isn’t the suspicious part. The behavior is. This is where hyper-activity scores reveal what the narrative never could.

Real Respondent

• “The gift giving and food”
  • Technology Threat: 12
  • Survey Activity Threat: 2
  • In-Survey Threat: 16

Fraud Respondent

• “The gifts and the family dinner”
  • Technology Threat: 28
  • Survey Activity Threat: 2,699
  • In-Survey Threat: 3

We probably don’t have to ask you if you want the respondent who attempted 2,699 surveys in the last 24 hours in your next survey data set…

Example 5

Tree decorating is one of the most wholesome, unmistakably human Christmas traditions… making this example particularly striking. The language is nearly identical across users, but the tech signals tell a different story.

Real Respondent

• “My favorite part of Christmas is the lights in the decorations and the food”
  • Technology Threat: 20
  • Survey Activity Threat: 3
  • In-Survey Threat: 0

Fraud Respondent

• “My favorite part of Christmas is the lights and decorating the tree”
  • Technology Threat: 42
  • Survey Activity Threat: 2
  • In-Survey Threat: 5

Both responses may feel authentic, but Defender flagged a tech threat for one of them. Fact is, a VPN alone might not be fraud, so low tech threat scores alone aren’t a huge issue. But, if it’s coupled with multiple layers of suspicious tech (as was this case) such as proxies, TOR, or obfuscation tools), or paired with hyper-activity or poor behaviors - then it’s worth considering weeding them out.

The Operational Danger

Fraud of this sophistication doesn’t just sneak through your OE review. It passes profiling, nails your screeners, and contaminates your dataset long before anyone notices something is off.

The examples above expose a critical shift: fraudulent respondents now blend seamlessly into the emotional and narrative fabric of open-ends.

In several cases, they even outperformed real respondents on spelling and structure. That means the industry’s old instinct-based OE review, “does this look human?” is no longer a viable safeguard.

Rep Data’s Counter-Approach

These examples make one thing clear: open-end reviews alone aren’t enough.

Rep Data’s Research Defender uses multiple layers to stops fraud such as:

• High-volume survey grinders (Example 4)
• Sophisticated tech-masked operators (Examples 2, 3, 5)
• Open-ended text that appears human but isn’t (Example 1)

See how we do it here.

Learn more about Research Defender.