Seven Mistakes to Avoid During User Interviews
Everyone has been in or conducted a not-so-great interview, wishing we had some golden rule book to have helped us through the process. As UX Practitioners and as basic human beings, we learn by doing and by example. Without the mistakes, how will we grow? Without the guidance, how will we know what we can achieve? UX Designer Frankie Kastenbaum and I teamed up to make this golden rule book (or maybe more of a golden list of guidelines, but that name isn’t as fun), and we did so in a UX-meets-Social Media manner. We observed how our peers, teammates, and even students were conducting interviews, while also researching the feedback from our instructors, mentors, and teammates on our interviews: from yesteryear to yesterday. From all of this, we want to present to you The 7 Mistakes to Avoid During User Interviews.
1. Asking more than one question at a time
Humans only remember a few things at a time, and broken up at that. Our phone numbers are broken-up groupings of 3–4 numbers for the sole reason that we’ll remember them. In a user interview, if you ask a string of questions at once, not only does it give away your follow-up answers before you ask them, but there is a high chance that the interviewee will not remember what they are being asked. With so many parts, or with so much time passed, just asking the questions in that way may lead the user to be more concerned with hitting all parts of the question than they are with the thoughtfulness of their answer.
Instead, ask the topic-setting question first — like, “Have you gone to a coffee shop in the past week?” And then asking, “What did you order?” and “How did you place your order?” consecutively afterwards, instead of asking something like, “Have you gone to a coffee shop in the past week? Did you stay there or take it to go? What did you get and how did you order it?” Think about how you would feel being asked all of those questions at once — would you remember all of that?
With that said…
2. Hammering questions at an interviewee
You want this interview to be a comfortable and safe environment, as if it were a conversation. With conversations come stories, and people tend to explain things better through story. So, asking your questions in an order that would tell a story would allow your interviewee to tell their story. Mapping out the ideal flow of your questions, with the expectation it could still pop around depending on the answers, you allow your interviewee to think and talk in a more consecutive way than as if they were a search engine. Asking a question, receiving an answer, then asking the next question can be unsettling — like you’re not valuing what the interviewee has to say and you’re just trying to check boxes.
With that being said, you still want to be cautious of making the interview too conversational that it loses its scientific and professional purpose. It’s one thing to make comments like, “I understand,” or, “that sounds frustrating,” but it’s another thing to engage in the conversation. For instance, when someone is talking about their experience in not being able to split a bill without frustration at a group dinner, you might want to chime in and talk about it. However, chiming in with your insights and experiences can impact how the interviewee answers going forward — they could start answering around your contribution when they should be answering on their thoughts and experiences. Something you can do in place of that is, “I understand. Can you tell me more about why you feel this way?” or blending the, “I understand,” into the next question you want to ask.
Which brings me to our next point…
3. Not asking additional follow-up questions
Sometimes we ask people what we think is an open-ended question, like “How was the drive?” and we don’t get the amount of information like we were hoping for. They may respond, “It was fine,” when we may have been expecting something like, “We left late but we ended up hitting every light!” When you get an answer like, “it was fine,” when you were hoping for more of a story, don’t give up and move to the next question. Instead, try asking something like, “Can you tell me more about that?” or “How was it fine?” Most often, the best information can come from asking the interviewee to further explain what they said. Plus, if they go down a rabbit hole, they can provide you with even more information, but it cannot be fully appreciated if the interviewer does not dive into the follow-up of these answers.
If you want to avoid too much follow-up, like “Can you tell me more about that,” review your discussion guide and try to find ways to make some questions more open-ended if they are not already. Questions that could benefit from these edits include ones like, “Do you eat popcorn at the movies?” You can reword it to: “What type of food do you eat at the movies?”
More questions that could be edited are ones that ask something like, “Do you drive or take the train to work?” Just making sure to ask why after they answer and making sure the “Why?” is in your guide will help remind you to ask. Having those reminders will help you to remember to always ask the why/how to an action or feeling.
Sometimes, though, assumptions can get in the way of asking why or how, but…
4. Assuming that since you and the interviewee do the same things that you know their reasonings
Remember that you are not the user! And remember what they say about assuming…
POV: You ask your interviewee what they did at the gym last week, and they tell you that they went to lift weights. You also go to the gym to lift weights, so you may not feel compelled to ask why they do, too.
However! Just because your interviewee also lift weights does not mean that you both lift weights for the same reason. It is very important to ask why even when you may suspect the answer — but remember that if they do not say it, it is not data! All you honestly know about your interviewee is what they tell you. Exploring the why on the questions and answers you find simple could open the door to so many possibilities.
Often the space you’re exploring has a different overlying issue from the one you started with that you cannot see without this conversation — so, in this case, you might be exploring why people at this gym are not lifting the heavier weights, so an assumption may be that that people don’t want to try too hard in front of other people. This user says they go to the gym to lift weights, and so do you, so you might assume that they go to build muscle. But what if they don’t? So you ask why, and they tell you that they are trying to strengthen a weak or injured muscle by lifting the small weights. This could help you and your team realize that the reason people may not be lifting the heavy weights could be that most of the visitors are going for these reasons and not to be the next Arnold Schwarzenegger.
That being said, maybe they do go for the same reason as you, but make sure to keep that to yourself…
5. Telling an interviewee whether their answer was correct
In some instances, when an interviewee is asking, “Is that what you meant by that question?” you can answer in a way that doesn’t tell them that they’re wrong if they are, but their perception of your question could also give you valuable data. However, if you need to ask again, work on rephrasing in a still-nonleading way to make sure you get your message across. But, when an interviewee asks, “Is that right? I could be wrong, please let me know,” you shouldn’t give much input on whether they were right.
A quick anecdote — I was observing an interview as a project manager for one of my research projects, and I witnessed the interviewer ask our interviewee what their current understanding of something was. The interviewee answered nervously and asked, “Is that right?” to which the interviewer responded, “I think so! That’s what other interviewees have said, so I think you’re on the right track.”
What the interviewer should have said in response to that question was something like, “There are no wrong answers, especially about your understandings. Any answer you give us today is tremendous help to our research and design.” It is important to remind the interviewee that there are no wrong answers and that all information about their experience is helpful. Interviews are conducted so UX Practitioners can better understand the users and the research focus.
Another thing that this interviewer should not have done is share with the interviewee what other interviewees have said, especially when we make sure to tell our users that everything in the interview is confidential and is to be shared only within the team. Even though no names were mentioned and no specific answers were shared, it not only breaks confidentiality but it also could lead to the interviewee then answering in a certain way that would not have if they had not known what other people were saying.
Telling the interviewee they were right or wrong could impact their future answers, just like this next point…
6. Asking solutioning questions, like “Would notifications make it easier for you?”
One of the first things I learned as a UX Researcher is to take users’ design recommendations with a grain of salt. People know their problems better than their solutions — if they know the solutions so well, why are they still experiencing the problem? It’s important that the UX Practitioners are the ones on the outside of the problem at hand to be an overseer and evaluator. It’s why we’re brought to the job: to help solve the problems. We’re here to solve pain points through user research and figure out what practices we can use to improve that situation.
The other thing I immediately learned as a UX Researcher is to not solution in the research phase. So to even bring up solutions like notifications is already something you shouldn’t be doing because it’s too early to be discussing that aspect. But we also should not be using our research methods to prove our assumptions and ideas for the project. Using leading questions like this not only increases biases, but also forces interviewees into answering questions in a mindset they were not in. They are there to discuss their experiences, not ideate the solution with you. These types of questions remove the ability for interviewees to get lost in their stories of an answer.
Speaking of getting lost in stories, you can encourage your interviewees to get lost in their answers with out next point…
7. Filling in the [awkward] silence
Silence is golden — it’s uncomfortable, but that’s the beauty of it. It allows the interviewee to explore their thoughts more and to share more of them because they want to fill that silence.
When you ask why or how, sometimes you can still get a short answer. You can use this opportunity to write down their answer, tell them you’re taking notes, and that silence may encourage them to share more of their thoughts. You can also nod along and show that you’re listening, but not vocally responding. This could encourage them to keep going. No one likes sitting in awkward silence, right? Use it to your advantage! It’s uncomfortable at first, but with more interviews under your belt, the easier it gets.
At the end of the day, user interviews are just conversations. I like to treat them like a conversation with a coworker at a work event — asking them questions and making sure that I don’t get too personal on my end because I personally don’t like mixing my work and personal lives together. It’s also important to, even though you’re not the user, to put yourself in your interviewee’s shoes. How would you respond to your interview style and methods? How would you respond to the phrasing of some of the questions? It is important to remove yourself from the user, but remember to keep empathy for them in their situation and their experiences.
Keeping these 7 aspects in mind have helped Frankie Kastenbaum and myself grow as interviewers and UX Practitioners, and hopefully they can help you in your growth, too.
Happy interviewing!
