Photo by Mohamed Boumaiza

Remote Usability Testing

Usability testing is used to evaluate the functioning of a website or other programme and to ensure that users can traverse it effectively. Usability testing is carried out by assigning users a series of tasks and evaluating their performance on those activities.

Before we get into the specifics, it’s important to understand when to do a usability test and what kind of test to conduct. Usability tests may be conducted on projects that have previously received a solution. These solutions may be as simple as wireframes, mockups, interactive prototypes, or they can be completely developed software or products already in existence. If your project is not yet at this level, you may do user research using alternative methods such as one-on-one interviews, focus groups, or surveys. Usability tests are classified into three types:

• Moderated vs Unmoderated
• Remote vs In person
• Explorative vs Comparative

‍

Plan:

Planning a USABILITY TEST is critical to the whole process since it determines what we’re going to do, how we’re going to do it, how many users we’re going to employ, the metrics we’re going to use, and the creation of tasks.

Planning a Usability test should include:

• The degree to which you are evaluating a website, a product, or any other application (for example usability test for account management in an application, usability test for home page in a website, usability test for a registration process in an application or in a website)
• Determine the usability test’s objectives (for example can user sign up with our application easily or you can be more specific like will the user be able to find the account settings option easily). Each page on your site may have a unique context from which we may derive a unique set of goals; thus, stay within the scope of the test and describe the objectives; these objectives will assist you in developing task scenarios.
• Choose your moderator testing type from Concurrent Think Aloud (CTA), Retrospective Think Aloud (RTA), Concurrent Probing (CP) and Retrospective Probing (RP).
• Choose your task participants carefully. Participants may be chosen based on the application under test. For example, to conduct tests on applications such as Instagram or WhatsApp, we can recruit participants between the ages of 18–30, as these age groups are more active on these products. However, if you want to learn how parents or older adults interact with these applications, you must select participants accordingly. With regards to the number of participants, you can choose a number greater than or equal to 5 and less than or equal to 8, as testing with fewer than five participants will produce inaccurate results, and testing with more than eight participants will produce nearly identical results to those obtained with eight participants.
• Typically, usability testing are performed using computers set for recording using OBS (Open Broadcaster Software), which includes audio and video capture devices. However, you may administer the exam remotely by using a video conferencing tool such as Zoom Call, Teams, or Google Meets and recording the whole test while moderating them on the call.
• Create a pretest or background questionnaire, a post test questionnaire, and follow-up questions to ask participants after they complete each task. Include quantifiable data such as successful task completion, mistakes, assistance, and time required to accomplish a task, as well as likes, dislikes, and suggestions.
• Include the equipment and apps that will be used to perform the test, such as the kind of computer operating system and the video conferencing platform (such as Teams or Zoom Call) that will be used to conduct the remote usability test. You’ll need a moderator and an observer to monitor and record the participants’ activities during the tasks.

Once you’ve prepared your test and ensured that it meets all criteria, recruit people by generating a recruiting form and posting or sharing it on social media or any other platform. Then, choose participants appropriately. Prior to conducting a real test, do at least one pilot test to determine how the test will go. Taking into account the user’s time, one test should typically take between 60 and 90 minutes; if a test exceeds 90 minutes, please notify the participant in advance so that he may arrange his day appropriately.

Before starting the test or calling a participant make sure everything is ready:

Consent Form, Pretest Questionnaire, Tasks List, Question after each task, Post test Questionnaire, Feedback Form, Metrics Sheets

‍

Test Process:

Begin by greeting the participant and expressing appreciation for their participation in the test; do not keep them waiting, as their time is valuable. Distribute the pretest questionnaire and encourage the participant to complete it while you try to initiate (maintain) a conversation in order to learn more about the participant and familiarize them to the setting.

Explain the procedure to the participant and give over the task sheet. Instruct the participant to begin the process; while completing the task, please record the time taken to complete it, the number of assists, and the number of mistakes. After completing each job, seek comments and record it.

After participants have completed all activities, request that they complete a post-test questionnaire and provide general comments. Appreciate the participant’s candid comments.

‍

Document:

You can report the results in CIF standard format which measures effectiveness, efficiency and satisfaction.

• Title Page
• Executive Summary — Give an overall summary of the complete report and include usability issues and redesign recommendation that you have suggested in the report.
• Introduction — Background : Give a background detail about the site or application that you are going to perform the test, usually it can be value in the market and how it is used by the users, Aim and Objective : State aim and objective of the test.
• Methodology — Overview : Give an overview of the methodology you are using to perform the usability test and metrics used for measuring, Participant demographic data : Tabulate the participant demographic data which you have collected from pretest questionnaire and explain in detail how you have chosen your participants, Insights from Preliminary Survey Data : Tabulate the data from preliminary survey that you have collected from pretest questionnaire and represent them in any graph format.
• Task — Task Scenario : Give information about the task and task description that participants have used to perform the test and tabulate the information about expected user time (you can get that from pilot testing) on each task and time limit on each task to perform (5* expected user time).
• Metrics — From the metrics collected while performing task measure the Effectiveness, Efficiency and Satisfaction and explain how you measured them. You can refer to this link to get to how to measure in detail https://usabilitygeek.com/usability-metrics-a-guide-to-quantify-system-usability/ .
• Equipment and Material — List all the equipment and materials that you used before, while and after conducting a usability test.
• Procedure — Tabulate the whole procedure you have followed while conducting a usability test from planning to evaluating the results.
• Results — Summary of Overall Findings : List all the findings from each task and give information about task completion rate, error & assist percentage per task and task completion average by the participants, Task Performance Data : Tabulate all the information related to task that is errors, assists, time taken to complete the task and the quotations or feedback that we get after completion of each task from the participant.
• Usability Issues — List all the usability issues and if possible, attach a screenshot of the website or product highlighting the issue.
• Redesign Recommendation — List the design recommendation that you have come up with that solves the above usability issues.
• References — List all the references that you have referred for information or gained knowledge.
• Appendices — Include all the forms and usability testing material that is used in conducting usability testing like consent form, post test questions form preliminary question form and their results as well.

Happy testing!