A survey is a powerful research tool, but it’s not all-powerful. Every tool has strengths and weaknesses. A hammer is great for roofing — not so much for mowing the lawn.
While a hammer’s form makes its function clear, tools like surveys take a little time to understand. The strength of surveys is in quantifying findings and extrapolating them to larger populations.
Let’s say a city wants to know how many people ride its public transportation systems. A survey with just a few hundred participants can tell the city how many people take its buses or subways within a 5 percentage-point margin of error. Particularly compared to a citywide census, a survey is fast and affordable.
So perhaps it’s no surprise that the 2016 GRIT report on research industry trends found that surveys — via phone, email, and mobile — are the most popular research tools for quantitative studies. What’s more, it found that quantitative studies are nearly twice as common as qualitative ones, which often rely on focus groups and in-depth interviews.
But it’s a mistake to think of these two methods — the quantitative survey and the qualitative interview — as interchangeable or mutually exclusive. Smart researchers recognize that they’re symbiotic. In fact, the same GRIT report showed that 68 percent of researchers use both methods.
How, exactly, do qualitative and quantitative methods complement one another? Qualitative data helps researchers understand their subject and formulate hypotheses, which then helps them craft better survey questions. Those surveys’ findings, in turn, can guide further qualitative studies in a methodological tête-à-tête.
Skirting Survey Mistakes
Just like any other tool, surveys can be strong or weak, useful or useless, depending on how they’re used. Focus groups offer an array of benefits that can help surveyors avoid four common snafus:
1. Using leading questions
Survey data can be extrapolated to broader populations only if the data points themselves are honest. Questions designed to convince, sell, or reinforce existing beliefs can’t accurately depict reality. Their answers may make you feel good, but in the long run, they’ll only end up hurting you.
Your goal with a survey is simple: Seek the unbiased truth. To do that, you need to create a survey with an objective in mind and hew closely to that objective. Using a pre-survey focus group helps you zero in on that objective.
When the D.C. United soccer team, an iResearch client, decided to rebrand itself, it wanted to learn which aspects of the fan experience were most important to its supporters. Prior to creating a survey, we conducted online focus groups to identify and prioritize fans’ concerns and gather feedback. By first talking with fans, we were able to create a concise survey that focused on the most salient and actionable items. Because we paired the methods, the team’s rebrand retained existing fans while exciting new ones.
2. Basing questions on assumed knowledge
Had we not first done focus groups for D.C. United, its survey would have focused on areas the team assumed were important. It likely would have excluded areas of greater importance to fans, resulting in wasted resources and irrelevant responses. The solution, of course, is to not assume. Base your survey questions on knowledge gathered through qualitative research, such as focus groups. Armed with knowledge of your customers’ values, pain points, and decision-making processes, you can create surveys with relevant questions and informed answer choices, reducing bias and increasing participants’ survey satisfaction.
Unfortunately, sending a survey based on assumed knowledge is often worse than sending none at all. Unlike focus groups, a survey is a fixed product. You can’t look at partial results or change the questions after it’s started.
3. Ignoring data’s relevance
When you’re creating a survey, you can include any question you want. But just because a question is interesting to you doesn’t mean you should put it in your survey.
A survey should address specific, actionable unknowns. If you fill your survey with questions that are merely interesting, the data you get back will be useless. Imagine, for example, that you’re launching a line of women’s lingerie. You could ask about everything from fabric to size to color to price, but if you don’t know why women buy lingerie in the first place, you’re no closer to knowing how to market and sell lingerie.
Instead of letting interest be your guide, first learn your customers’ “why.” If you can find out through focus groups the reasons why women buy lingerie, then you can use your survey to find out “how many” — in other words, which of those factors are most important. You can then target your marketing at the top reasons women lingerie consumers will want your product.
4. Including too many questions
When writing a survey, it’s easy to forget that no one will spend as long taking it as you will writing it. A survey that takes longer than a few minutes to complete is a pricey proposition.
Long surveys take time to administer, which makes them more expensive, and they annoy participants, which leads to high drop-out rates. To make up for the high drop-out rate, you have to conduct the survey across more people, which also makes them more expensive.
Instead of sending a novel of a survey, mercilessly cut questions until you’re left with only those that ask for the specific information you need and that can be best answered in a survey format.
Is the question merely interesting? Cut it. Is the question better answered in a conversation? Cut it. Does the question not require a large sample size? Cut it. Is it asking a “why”? Cut it. Is it asking, “How many?” Keep it.
Trust Your Tools
Does a claw hammer have a sharp edge? Yes, it does, just like your lawnmower, and it’s hundreds of dollars cheaper, too. But only one will cut the grass, and only one will repair the fence around your yard.
When gathering insights, it’s tempting to take shortcuts. Surveys and focus groups both involve questions, participants, and data. But only one will tell you “why,” and only one will tell you “how many.” And only together will they tell you what customers truly want from your product.