Mastering UX surveys: Best practices and how to avoid biases.
Introduction
Surveys are an essential tool for gathering valuable insights and feedback from users. By employing well-designed surveys, UX professionals can obtain quantitative and qualitative data, enabling them to make informed decisions and improve the overall user experience. In this article, we will explore the ins and outs of conducting effective UX surveys, including the best practices, potential biases, and additional resources to enhance your surveying skills.
When should I conduct a survey?
Surveys can be conducted all along the design process. For instance, conducting surveys in early stages helps in understanding user needs, preferences, and pain points. This initial feedback can inform the direction of the design process and to validate your hypothesis.
Best practices
1. Clearly define the goals and objectives of your survey.
Determine what specific information you are seeking to gather, whether it’s user preferences, pain points, satisfaction levels, or other relevant insights.
2. Identify and target the appropriate audience.
Ensure that the participants are representative of your user base or the specific target demographic you are interested in. This helps to gather relevant and accurate data.
3. Ensure your questions are phrased in a neutral manner
Avoid leading or biased language. It will help to elicit honest and unbiased responses from participants.
4. Utilize a mix of question types
Including multiple-choice, rating scales, and open-ended questions. This variety allows for a comprehensive understanding of user perspectives. In order to reduce survey fatigue and increase data accuracy, allow participants to skip questions.
5. Incorporate open-ended questions
To give participants the opportunity to express their thoughts and provide detailed feedback. These responses can reveal insights that might have been missed with closed-ended questions. E.g. What improvements would you suggest for our product?
6. Ensure each question is clear and concise
Focus on a single aspect or topic. Complex or multi-part questions can confuse participants and lead to inaccurate responses. Avoid unnecessary questions and focus on gathering the most crucial information.
7. Arrange questions in a logical and intuitive order
To ensure smooth flow and coherence in your survey. Begin with easy and non-sensitive questions to establish rapport before moving on to more complex or personal ones.
8. Avoid overlap or repetition
Specially when providing response options, to prevent confusion and ensure accurate data collection.
9. Design a good rating scale
Rating scale questions, such as Likert scales, are valuable tools for measuring attitudes, preferences, or satisfaction levels. They are also helpful to reduce the effect of the extreme answer and the central tendency biases.
10. Use industry standards
For example, The System Usability Scale (SUS) is a widely-used survey instrument for assessing the usability of software, websites, or other interactive systems. It can provide a standardized method to measure and compare usability across different products or interfaces.
Mitigating main biases
Biases are everywhere, and it is impossible to avoid them at 100%. That said, knowing these different biases will help you mitigating their effects, ensuring the accuracy of the collected data.
1. Acquiescence bias
Acquiescence bias refers to the tendency of participants to agree with statements or questions without giving them careful thought. It happens specially in controversial topics. This bias can skew survey results.
To minimize acquiescence bias, you can employ reverse-worded items in your survey. Instead of consistently using positive statements, incorporate negative statements that participants can agree or disagree with. This helps to balance the responses.
2. Extreme answer bias
It occurs when participants consistently choose extreme response options instead of providing nuanced or moderate responses. This bias can affect the accuracy of data analysis.
To counter extreme answer bias, you can include midpoint anchors on rating scales. By providing a neutral or middle response option, participants are encouraged to provide more nuanced answers rather than consistently selecting the extreme ends of the scale. You can also include an open-field in the questions, leaving room for people to explain themselves.
3. Central tendency bias
As the opposite of the extreme answer bias, the central tendency refers to participants’ inclination to select responses near the center of the scale, regardless of their actual opinions or experiences. This bias can flatten the data distribution and reduce its usefulness.
To address central tendency bias, you can use odd-numbered scales instead of even-numbered scales. Odd-numbered scales eliminate the midpoint, forcing participants to choose a response that aligns with their true opinion, rather than defaulting to the middle option.
4. Social Desirability bias
Occurs when participants alter their responses to align with social norms or what they perceive as socially acceptable. It also refers to the tendency for people to present themselves in a favorable way. This bias can lead to inflated positive feedback or underreporting of negative experiences.
To avoid it don’t make aspirational or future-related questions (what would you do in case “x” happens?) and use present questions related to actual facts (how many times did you did “x” the last week?). Also ensure anonymity and confidentiality of survey responses, so participants feel more confident to give their true opinions.
5. Attentional bias
Attentional bias occurs when participants focus their attention on specific aspects of a question, leading to skewed responses.
To mitigate it, you can use a randomized order, include different type of questions and use images and other elements as bold text to capture users attention.
Conclusion
By incorporating these tips and understanding potential biases, you can design and conduct effective UX surveys that yield valuable insights. Remember to apply best practices, employ a mix of question types, and keep the survey concise and focused. Through well-executed surveys, you can gather actionable data to drive informed decision-making and ultimately enhance the user experience.
Remember to consult the sources provided to further refine your surveying skills and make the most of user insights.
Sources
- “The Handbook of Social Research Methods” by Bickman, Leonard, and Debra J. Rog
- “Survey Methodology” by Robert M. Groves, Floyd J. Fowler Jr., Mick P. Couper, James M. Lepkowski, Eleanor Singer, and Roger Tourangeau
- “Conducting Surveys: A Guide to Survey Research” by Raymond M. Hyman and Charles D. Tanur
- “Quantifying the User Experience: Practical Statistics for User Research” by Jeff Sauro and James R. Lewis
- https://www.qualtrics.com/experience-management/customer/user-survey/
- https://uxplanet.org/this-is-all-you-need-to-know-to-conduct-a-ux-survey-50400af45920
- https://www.hotjar.com/ux-design/surveys/
- https://mopinion.com/user-experience-surveys/
- https://qualaroo.com/blog/ux-survey/
- https://userpilot.com/blog/ux-survey/
- https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html
- https://www.qualtrics.com/blog/three-tips-for-effectively-using-scale-point-questions/
- https://www.qualtrics.com/experience-management/research/question-sequence-flow-style/
- https://en.wikipedia.org/wiki/Acquiescence_bias
- https://www.risely.me/5-ways-to-overcome-central-tendency-bias
