Biased Survey - How to Avoid Biased Survey Questions

Gaining important insights through survey research is an efficient decision-making strategy. It’s a sensible way to make a good call when a difficult decision is required, one that needs to be based on relevant feedback from stakeholders like customers or employees.

However, there are many ways to err when surveying respondents, including biased survey questionsthat can completely ruin research and lead to incorrect conclusions.

Creating a survey is a task that should be taken seriously; data accuracy is always essential to good research. Luckily, if you want to become a seasoned survey writer who successfully avoids different types of survey bias, you’re in the right place.

If you are ready to collect reliable feedback from respondents, please keep reading.

Understanding biased survey questions: What is a biased survey

A biased survey includes questions or phrasing that confuse respondents and lead to inaccurate data because respondents are directed to answer in a certain way. This can be done purposefully or inadvertently by those authoring a survey.

There are many potential disadvantages of ignoring biased questions in your survey. The main one is that they waste time and resources because they often lead to useless, inaccurate results.

Creating a well-designed, unbiased questionnaire requires making sure questions are written clearly and straightforwardly, while also ensuring they align with your respondents’ knowledge and motivations.

Dealing with the biased survey: How to recognize biased survey questions

Unfortunately, there is no exact method to catch all the potential flaws while formulating your survey questions (at least not without the help of a seasoned researcher).

However, there are some indicators you should look for when wording questions. Biased survey questions are often of two types: respondents do not interpret the wording similarly or the wording influences respondents’ answers. Always read questions carefully, and more than once, before sending them out in a survey.

Knowing the most common types of biased survey questions is a good start. Learn how to recognize them so you can avoid biased survey questions in the first place.

Image Title

Photo by Celpax on Unsplash

1 Leading questions bias (with examples) - what are leading questions and how to avoid them?

Leading questions push the person taking the survey to answer in a certain way, with phrasing that contains the statement the researcher wants to pull out from respondents.

The key element of leading questions to be on the lookout for is that they often contain an assumption and tend to be overly definitive in acquiring desired information.

Examples:

Do you support the new tax proposal? (This question is biased because it implies that the respondent should support the proposal.)

More employees working remotely reported being happy compared to office-based workers. What do you have to say about it? (This question is biased because it implies that the respondent should prefer working remotely.)

To avoid leading questions, it is important to phrase your questions neutrally and balance the options for answering. A better way to word these questions might be:

What are your thoughts on the new tax proposal?

Would you prefer working remotely or in an office?

2 Assumptive questions

Assumptive questions contain an assumption about a respondents’ habits, preferences, values, etc. When answering these types of questions, respondents usually end up being pushed to agree or disagree with the statement included in the question.

Also known as loaded survey questions, assumptive questions not only prevent you from getting honest feedback, but also lead to a higher dropout rate (the number of respondents that choose not to finish).

Examples:

What is your favorite cocktail bar? (This question is biased because it implies that the respondent drinks.)

How much did you enjoy attending our conference? (This question is biased because it implies that the respondent enjoyed attending.)

To avoid assumptive questions, ensure you are not making baseless assumptions about your respondents. A better way to word these questions might be:

Do you have a favorite cocktail bar, and which one is it if you do?

Did you enjoy attending our conference?

3 Double-barreled questions (example) - what is a double barreled question in research

Double-barreled questions, also known as double-direct questions, are two questions (issues or topics) asked at once. The unenviable task for recipients is to answer the question with only one answer.

By splitting up one double-barreled question, you make your questionnaire less complex and respondents won’t get confused. The essential tip for creating an excellent survey—one that will produce a great experience for respondents—is to not overcomplicate your research design.

Example:

Do you think our products are innovative and practical? (This question is biased because the attributes “innovative” and “practical” are not synonymous.)

To avoid double-barreled questions, ensure that only one piece of information is sought per question A better way to get this information is to ask two separate questions:

Do you think our products are innovative?

Do you think our products are practical?

4 Ambiguous questions

Ambiguous questions can be interpreted in more than one way. They can confuse respondents and can easily lead to inaccurate research results.

Also known as unclear or vague questions, ambiguous questions don’t have a specific query, meaning respondents can answer the question correctly even if providing information about entirely different topics.

Examples:

How are you doing today? (This question is biased because respondents could understand the question as asking them how they are feeling or asking about their health.)

Have you bought anything in the past six months? (This question is biased because it is unclear and doesn’t reveal what the purchase was, where it was made, etc.)

To avoid ambiguous questions, you have to have the overall research goal constantly in your mind. Write brief questions that gather relevant information for your specific research topic.

Image Title

Photo by Brett Jordan on Unsplash

5 Dichotomous-biased questions: why are absolute questions not the best option for some surveys?

Dichotomous questions contain only two possible response variables. Often these are “Yes/No” or “True/False” questions. While they can be helpful to some researchers, they often tend to result in inaccurate research data.

Example:

Do you know how to play the piano? (This question is biased because it doesn’t allow the respondents to describe their piano playing skills. Instead, they end up picking an answer that may not resonate with them completely.)

To avoid the problem of dichotomous questions, try using multiple-choice questions or a Likert scale instead. A better way to word these questions might be:

How well do you know how to play the piano?

  1. Not very well
  2. Average
  3. Very well

6 Survey Questions that include jargon language (examples)

Jargon is a word or expression used by a group that is difficult for people outside that group to understand (such as legal jargon or sports jargon). When used in a questionnaire respondents may not understand what is being asked.

If the jargon is specific to a particular field, respondents may also become frustrated, leading to a higher dropout rate. Finally, if respondents misinterpret the questions, this can lead to inaccurate responses.

Example:

Are you satisfied with our website’s UX? (This question is biased because the question is probably inadequately worded for a wider population or your target audience.)

Here’s a quick tip. To ensure you’re not using jargon, find a few friends or colleagues with different demographic profiles and see if they understand the questions.

How to avoid biased survey questions and, ultimately, a biased survey?

When designing survey questions, word choice may change the research results entirely. To gather reliable data, questions need to be clear and concise.

If you don’t have experience in this field, then the best way to a well-written survey is to “steal” our best practices to avoid biased survey questions. You can browse through our survey examples or SurveyPlanet’s helpful blog posts. Our idea is to make survey research an “easy peasy” task, one that anyone can do professionally. Start your survey-making journey by signing up today!

Photo by Hasan Almasi on Unsplash