20 tips on how to run better usability testing sessions
Before getting into the tips themselves, let’s go back to the basics:
What is usability testing?
Usability testing is a method of testing the functionality of a website, app, or other digital product by observing real users as they attempt to complete tasks on it. The users are usually observed by researchers working for a business. Mainly, usability testing is all about observing real people interacting with your product.
Why is usability testing important?
Usability testing is done by real-life users, who are likely to reveal issues that people familiar with a website can no longer identify. The goal of running such sessions is to reveal areas of confusion and uncover opportunities to improve the overall user experience.
Even the best UX designers can’t design a perfect — or even good enough — user experience without iterative design driven by observations of real users and of their interactions with the design. There are many variables in designing a modern user interface and there are even more variables in the human brain. The total number of combinations is huge. The only way to get UX design right is to test it.
What do you need to run a usability testing session?
The core, necessary components that are needed for usability testing are the facilitator, the participant, the product to be tested, and the tasks that should be performed.
The facilitator guides the participant through the test process by giving instructions, answering the participant’s questions, and asking follow-up questions.
The participant should be a realistic user of the product or service being studied.
The tasks in a usability test are realistic activities that the participant might perform in real life. They can be very specific or very open-ended, depending on the research questions and the type of usability testing.
What type of usability testing methods are there?
Usability testing can be moderated or unmoderated. In moderated usability testing the facilitator still interacts with the participant and asks them to perform tasks, while in unmoderated usability testing, the facilitator does not interact with the participant, the researcher only gives the participant the task that should be performed.
The usability testing can take place remotely or in person. Remote usability tests are popular because they often require less time and money than in-person studies.
How many participants are needed for a usability testing session?
The goal of usability testing is to understand what problems people have with the design as they go through the product, trying to complete certain tasks. What happens, in this case, is that, often, the same problems start to repeat, again and again, making it clear that this is a problem for everyone. There might be problems that can be seen right at the beginning of the study, even after one or two participants. If that is the case, there is no need to test further. Usually, after testing with 5 or up to 8 users, the majority of the problems can be identified.
By carrying on testing more participants, there might be identified other problems, but the most crucial problems have been identified previously and there will definitely be diminishing returns on the extra ones.
What is the process flow for usability testing?
Planning the usability test
Planning the details of the usability testing session is, in some ways, the most crucial part of the entire process. The decisions you make at the start of the testing process will dictate the way you proceed and the results you end up with.
In this stage, you should define:
- What are the areas or problems that are in focus and what are the goals of the study?
- What profile of users is needed. Typically, these are representative of your user personas.
- Script for the questions that are going to be asked and the list of tasks that should be performed on the prototype.
Setting up the test environment
In this stage of the process, the logistical details of your usability testing sessions should be decided:
- Location — is it a remote or in-person session? If it is remote, what app are you going to use for the session?
- Timetable: agree on a schedule with the participants for the session.
📢 Advice #1: Do not schedule more than 3 sessions per day. A usability testing session requires a lot of mental effort from your side. As a facilitator, you need to focus on every little thing that the user does and on all of their non verbal expressions so that you can always come up with the right follow up question.
📢 Advice #2: Do not schedule back-to-back meetings for the usability sessions. Sometimes, the sessions take more time than planned. Make sure that you don’t have to rush the participants and that you have time to finish all of the tasks and to clarify all of your questions.
📢 Advice #3: Allocate half an hour for reflecting on the sessions with the team right after the test. That way, all of your ideas and impressions are fresh in your mind. Write all of them down and briefly discuss them with the rest of your team that followed the session.
- Moderators: who will run the testing sessions?
- Note-taker: who will be taking the notes during the session?
📢 Advice #4: Have one of your teammates help you with the notes. That way you can fully focus on the discussion.
- Recording setup: recording testing sessions gives the chance to review them later and catch all kinds of data that the note-taker might miss or not have time to record.
💯 Take it one step further and live stream your test sessions to your colleagues in another room. You may have tried this in a Design Sprint. Not only will you get multiple perspectives on each test session — you will also significantly speed up and improve the analysis process but also make the rest of the team feel involved in the process.
📢 Advice #5: Make sure that you test your recording setup before to ensure that it works as desired. It would be such a shame not to have video or audio of your session, wouldn’t it?
- Collect all this information in one centralized place and use it as your main guide towards the next steps: recruiting participants and designing the actual session.
Recruiting participants
Whom you recruit, and how, depends on your testing goals and your budgetary constraints. You can recruit from the pool of users that are already using your product or you can use online websites that offer users that correspond to your criteria. You can always be creative in how you recruit the participants for the study.
📢 Advice #6: Always compensate your participants for their efforts. Depending on the type of relationship you have with the participants (for example if the participants are from a company that collaborates with the company for whom you are working with, or if there are colleagues from your workplace, it is not recommended to offer money as a compensation) you can compensate them with cash, gift cards or any other type of gifts that are appropriate.
📢 Advice #7: In order to make sure that you are well prepared and you didn’t miss any details, you can go over the checklist created by Steve Krug, the writer of “Rocket Surgery Made Easy”, one of the best books on the topic, alongside with “Don’t make me think”.
Conducting the test
Here you will run your usability test with your participants.
📢 Advice #8: Always run dry tests beforehand with your team. Dry tests are an excellent opportunity to identify there will be any problems in collecting and recording data (recording setup) and will create the opportunity to uncover any issues with your product or your script before your participants do.
After the session is over, there are 2 more additional steps:
Analyzing the data
All the data gathered from your usability test (here the notes and the recording will be used) will be analyzed in order to get critical insights and actionable intelligence on how to improve the product’s usability.
Reporting the results
Share your usability tests’ findings with relevant stakeholders across your team, to earn buy-in to proceed with your suggested improvements.
How to conduct the interview:
📢 Advice #9: Make your participant feel comfortable
Always provide a nice welcome for your interviewee: greet them by name, offer a drink (if possible), and initiate friendly small talk before moving to the main topic of discussion. Start the conversations by learning a little bit about the interviewee. This information can not only be used for future analysis but it is more of an ice-breaker and a great way to get some context about your interviewees.
📢 Advice #10: Make sure that the participants understand the purpose of the interview and what is expected of them.
Make sure you explain the purpose of the interview to the participant. Describe what you are trying to achieve and explain how you plan to use the results. The reasoning behind this is to give your participant an idea of what sort of questions they might be asked so that they don’t feel confused during the interview.
Never skip this step. As you are so familiar with all of this information, you know your product front & back, you might experience what social psychologists call “the curse of knowledge” — the cognitive bias that makes it difficult for you to remember what it’s like to be a beginner seeing the content for the first time. So, even if you are excited about getting straight into the testing part, it is crucial that you lay a good foundation by explaining to the participant what is the product that they are testing, what is the scenario that they should simulate, and what is the goal of the session.
📢 Advice #11: Explain the structure and the duration of the session so that the participant knows what to expect. A good example is presented below:
“The session will take about 60 minutes which I will break into 2 parts. In the first part, we will take a look at the application where I will ask you to perform some given tasks. In the second part, I will ask you some general questions regarding the system that you experienced.”
📢 Advice #12: Clarify from the beginning that this is not a test of their competencies and there are no bad answers. If during the interview, any prototype or product is being tested, make it clear to the interviewee that you’re testing the prototype, not them.
📢 Advice #13: Ask for permission to record the session.
“With your permission, I am going to record what happens on the screen and our conversation. The recording will only be used to help us figure out how to improve the app and won’t be seen by anyone except the people working on this project. It also helps me because I don’t have to take as many notes.”
📢 Advice #14:Remind your participants that they are expected to talk out loud, expressing all their concerns, expectations, thoughts, and feelings while they carry out the activity with the system. In this way, the evaluator can obtain valuable insights into how the users operate the system and what their strategy is for carrying out the tasks.
📢 Advice #15: Listen and ask follow-up questions.
“The foundation for conducting a good interview is to keep an open mind and be truly curious about the participant’s perspective.“ You are there to learn something from your interviewees not the other way around!
Listening carefully to what the participants are saying and dig deeper to learn more about interesting behavior and insights. The better you listen, the better data you can gather. Always try to dig deeper to understand the interviewee’s point of view by asking follow up questions or asking to elaborate on their responses. If there is any unclear information that the user is providing, ask for clarifications. Do not leave those questions to the end of the interview session, because it will be hard to recreate the original context.
📢 Advice #16: Use the ‘Five Whys’ technique.
By repeatedly asking the question, “Why,” five times you can discover a lot and get into the cause of the problem. The whole idea is to remove the layers of the problem and discover the root cause of it.
📢 Advice #17: Summarize the information received from the participant.
Another technique you can use is, after each chapter or section from your interview, summarize the information received from the participant. This way, you show interest in what they are saying and you can check if you really understand the point they are making. If you simply repeat what they just said, people tend to add more details to it.
📢 Advice #18: Make it feel personal
Usually, it is difficult for your participants to give you the answers you are looking for. A method that can be used in this case is the critical incident method in which you will ask the participant to recall a specific situation from the past in which they faced a terrible or really excellent experience. Unlike general situations from the past that the user might easily forget, the extreme cases are often more vivid in users’ minds, meaning they can remember some specifics that will be valuable for you. This is why the critical incident method is especially great for exploratory interviews.
Example: In a work environment, you can ask your participant to describe a problem that they faced at their job and how they solved it.
📢 Advice #19: How to close the interview
You should finish the interview with a wrap-up summary, which will give your respondent a sense of closure. Always ask them if they have any questions or thoughts connected to the topic which you had not raised. You may collect ideas to develop for the next interview. After that, thank your interviewees for taking their time to participate in the study.
📢 Advice #20: Retrospective
Allocate time right after the interview for a retrospective session with the rest of the team. It is good to discuss the findings while they are fresh in your mind. Also, this is a good time to reflect on how well the interview went and what you can do to improve the quality of interviews in the future.
Usability testing is an amazing tool for all in the UX industry and it should be embraced as early and often as possible in a project. Even though it might not be perfect the first time you do it, the key is to keep learning and practicing for the best results. Like everything in life, practice makes perfect.
Happy and fruitful testing! 🚀
