Survey Length: How Long Is Too Long?

Steven Snell, PhD
SVP, Head of Research

If you’re worried about your survey length, it’s with good reason. Decades of research show that respondents are more likely to quit long surveys, and those who finish give lower-quality responses. Neither outcome gives you the data you need to make business decisions, so you're left to ask, "How long is too long?"

The simple rule is to keep your survey as short as possible. Nevertheless, a survey can be too short. If you rush to gather responses only to discover that you don't have the right data to answer your business questions, you've done a disservice to yourself and your respondents.

FOUR QUESTIONS TO DETERMINE THE RIGHT SURVEY LENGTH

We might like a rule about survey length – don't ever go past three minutes, or seven, or 15. However, respondents sometimes abandon even the shortest surveys or finish very long ones. There’s a lot more that goes into survey length beyond the minutes it takes the average respondent to finish answering questions; instead, many considerations impact how long your survey should be. Here are four questions to answer to determine the right survey length.

1. How motivated are respondents to participate?
   Not all respondents are created equal. Some enter your survey excited about the topic. Others come with a healthy dose of skepticism, uncertainty, and even resentment. If you are surveying highly competent and motivated respondents, for example, interviewing procurement experts on their processes, they will likely be engaged and willing to respond to a few extra questions. On the other hand, if you are talking to a grumpy or churned customer, you should consider yourself on borrowed time and ask just the essential questions.
   Takeaway: If you have motivated respondents, you can ask more questions.
2. Are you compensating respondents?
   Much market research is conducted with standing panels of research participants who agree to take surveys for money or other compensation. In other circumstances, researchers invite customers or other people known to them to take a survey, usually for money. Generally speaking, when surveys are longer, respondents should receive more money. Nevertheless, fewer respondents are willing to take long surveys, even for a higher incentive. Worse still, high incentives can be a fraud magnet such that real respondents may turn away from a lengthy survey, but fraudsters may gladly line up for the higher payout.
   Takeaway: If you are using incentives to drive participation, you can likely get away with more questions, but you should still be mindful of asking an inordinate number of questions.
3. How are you distributing your survey?
   How and to whom you distribute your survey can help determine the appropriate length. Phones are everywhere, and with work email increasingly on phones, even B2B surveys are more likely than ever to be completed on a phone. While the convenience of phones may yield a higher response rate (i.e., people starting your survey), evidence suggests that surveys take longer to complete on a mobile device, resulting in a lower completion rate. Similarly, web pop-ups and SMS surveys should still be shorter. Generally, five questions or fewer are best as respondents want to avoid a prolonged interruption from what they were doing before being intercepted by your survey.
   Takeaway: If you expect significant numbers of mobile respondents, keep your survey shorter.
4. How easy or complex are your questions?
   Finally, survey design matters! A well-designed "long" survey may take much less time to complete than a survey with many fewer but more complex questions. Beware of multiple open-ended questions, grids/matrices that require multiple decisions, ranking exercises, and questions that probe abstract and complex concepts. Avoid questions with long lists of nominal response options (e.g., factors, issues), opting for scaled responses where possible.
   Takeaway: If your questions are straightforward and easy to answer (e.g., single-response questions with scaled responses), you can likely ask more questions.

START WITH THE END IN MIND

If you're unsure how to design a compact survey, consider working backward from what you need to learn from the data – start by drafting a mock report of your findings and then write the survey questions that would allow you to fill in that report. If you write every question with your analysis in mind, considering the motivations and abilities of your respondents and the distribution method and complexity of questions, you are more likely to craft an appropriate survey.