Make better surveys
There are many ways to conduct user research, but the one that we see most often as a user is probably the survey. They come in many different forms, from an in-depth questionnaire to an NPS scale when you pass level 2 on that mobile game.
Surveys are a great way to collect quantitative (quantity) data that answer questions such as “how many” or “which”. This makes them great tools to validate your primary research findings and assess whether your conclusions will fit the broader userbase.
But, it can be deceptively tricky to use surveys effectively. Getting answers is fairly straightforward, but crafting questions that get the right answers is much more challenging.
Following, are some best practices you can refer to when you’re crafting your next survey to ensure that you’ll get more insightful information that will lead your project to greater success.
Short and sweet.
Most research methods allow us to offer participants incentives that encourage them to give us their time and attention. And although it’s really helpful that surveys are cheap (or free) to make, it also means that people won’t have much incentive to complete long and laborious surveys—unless they have strong opinions to voice perhaps. Therefore, it’s not recommended to depend on a single survey to answer all of life’s questions.
Write down the main learning goals that you need to answer in order to progress to the next phase, and then carefully craft no more than 20 questions aimed at answering those learning goals.
It will be much more effective if you make the appropriate changes in your designs after you synthesised your findings, and then run another study to validate those changes before moving on.
Tip: Encourage your intended audience to participate by telling them how long the survey will take when you recruit them.
Be personable
Consider your survey a good conversation. Would you go up to a stranger and immediately bombard them with personal questions? They would probably get really creeped out by that approach.
We often start with an icebreaker question and then go deeper from there. Similarly, we should try to keep our demographic questions till the end of the survey once we’ve built some rapport with the participant. Most demographic questions can be answered with some secondary research, but if there is missing data or if it’s an especially personal question, people will be more likely to answer it at the end.
Spell it out
In an interview study, we can try to read people’s body language in order to understand how they feel and help them if they get confused. With a survey, however, that won’t be possible.
Therefore, we need to carefully craft our questions to ensure that they will be easy to understand without the need for context or further clarification.
For example, you might ask participants, “Do you often use your webcam on Zoom calls?” But what exactly do we mean by that?
It can easily be rephrased as: “Do you often use an external webcam to make video calls?”, “Do you turn on your webcam when receiving a Zoom call”, and many other interpretations that are all acceptable.
You might expect your interpretation to be the “obvious” one, but without an understanding of the context and subject matter, it’s open to other people’s subjective interpretations which can potentially cause you to draw the wrong conclusions.
Therefore, read your questions again and assess how much subject matter knowledge or assumptions are required for them to sound like your interpretation.
Never say “and”
Participants can’t explain their answers either. In a survey, most answers are predetermined by you, and even if a text field is available, people are not likely to elaborate much on their answers. As a result, it will be very difficult to get a clear answer if you ask double-barreled questions.
For example, “Do you believe that Art and English should be compulsory subjects in school?” is in fact two separate questions.
People might feel that Art doesn’t have to be compulsory, but might believe that English should be. Unfortunately, they are grouped together and it will be impossible to know what people really believed with a simple binary answer.
Thus, don’t just look for magic 8-ball responses to your questions. Try framing your questions in a way that will give you as many unique insights as possible.
Don’t assume
The golden rule in any research study is to make sure that you don’t ask any leading questions.
You might ask people, “What did you recently make with the collage feature in this application”. But, this question is based on several assumptions about your participants that might cause you to miss out on valuable insights. Who is to say that they even knew about this feature or that they were able to successfully use it before?
Start by ascertaining what bias you have about your participants and what is an assumption that you have made about them. Then, go over your questions and look for areas that show signs of these assumptions and biases.
We shouldn’t require a lot of pre-requisites before people can answer a question. Surveys should be accessible to users with a variety of skill levels and backgrounds. So, try to break the more complex questions up into several smaller parts to get a more detailed answer and lead your participants to your intended interpretation.
Final thoughts
Making the most of the survey platform’s formatting features will help you a lot. We need to reduce participants’ cognitive load and make it easy for them to answer the questions. Therefore, consider what will be the most appropriate input method for your answers. Google Forms is a free survey tool that allows people to use a variety of input methods like text, radio buttons, and checkboxes. This will help you to give participants a more delightful experience and synthesise their data more easily.
Lastly, after you finished your questionnaire, remember to run a pilot study with a colleague or friend. We should make sure that everything works as expected and test how other people might intemperate our questions while there is still an opportunity to make some changes.
Please follow me on Twitter for more design and tech-related content, or learn more about me through the links in my bio.
