How we avoid bias in our surveys
/
Almost all our pricing packages include a survey portion. That’s because surveys serve as the crux of your content campaign. Once we have the data, we can use it to create numerous high-performing content assets, including beautifully-designed reports for lead gen.
Bias shows up more often in surveys about people’s beliefs, values, and lifestyles. It isn’t as present in business surveys — there’s no good reason to lie on a question like, “What is your company’s annual revenue,” for example.
Nonetheless, as with any survey, bias is always a concern. Here are a few things we do to ensure your data is solid and actionable.
We Draft Strong Survey Questions
We don’t just pull survey questions out of nowhere. We work with you extensively before launching your survey, so we can draft questions that encourage respondents to answer truthfully.
We also avoid common pitfalls like:
Leading questions
Assumptive questions
Confusing questions
Pushy questions
Ambiguous questions
Double-barreled questions
Instead, we focus on drafting questions which are relevant to the survey sample, easy to understand, and easy to answer.
Good survey questions are essential for avoiding response bias, which refers to people’s tendency to answer survey questions untruthfully or misleadingly. Bias can also creep in if your survey questions are confusing.
For example, you could ask a question like “Are you satisfied or dissatisfied with the pay and benefits of your current job?” But this might put some respondents in a difficult situation. What if they are happy with their pay but unhappy with their benefits? Did you make it clear that the survey is anonymous — what if your respondent is worried their employer might read it?
One market researcher gives another great example:
“A satisfaction survey may ask the respondent to indicate where she is satisfied, dissatisfied, or very dissatisfied. By giving the respondent one response option to express satisfaction and two response options to express dissatisfaction, this survey question is biased toward getting a dissatisfied response.”
Even subtle problems in your questions can tilt the data in your survey, which is why we keep all our questions straightforward.
We Target the Right Respondents
Sampling bias can be introduced into your survey by your selection of individual respondents or groups of respondents. If respondents aren’t properly randomized, or if your survey criteria aren’t accurate enough, you could receive skewed responses or responses that are unrepresentative of your target audience.
Let’s say you want to run a survey of decision-makers at enterprise-level software companies. If your sample selection includes 211 respondents who run startups and only 89 enterprise-level executives, you won’t be able to base your analysis on the precedent that you surveyed enterprise executives.
A statement like, “73% of enterprise-level executives say they aren’t prioritizing new technology investments next year” would be misleading unless you specify that the “73%” you cite only refers to the 89 enterprise-level executives who represented only 29.7% of your survey respondents.
Sometimes it helps to expand your criteria and get a wider range of responses.
Nonetheless, to avoid selection bias, we only target respondents who are qualified to take your survey based on your criteria. We also ask screening questions to determine the roles of respondents in your sample, then present this information to the audience for full transparency.
We Rely on Survey Experts
All our surveys, including our interview response surveys, are run by a partner that has years of experience surveying business professionals. We’ll work with you to draft questions, then liaise with our survey partners to ensure your survey is well-executed and productive.
To learn more about our survey capabilities and our partnership, talk to an analyst today.
