UX Research
Bring back the expert UX review
Most of the time, the designs I test with users, could have benefited from a UX review at an earlier stage.
Somewhere in the evolution of user-centred design (what we called it before UX), the humble UX review seems to have fallen out of favour. I get the best results with my clients when I regularly review designs in addition to carrying out user research.
In this post I’m going to talk about the pros and cons of this method and why I feel the approach should be used more often.
What is an expert review?
In an expert UX review, an experienced UX practitioner, applies the understanding they have gained from past user observation, to give the designer advice about their designs. They look through the designs, spot the weaknesses they see and then give recommendations for how the usability might be improved.
If you want an idea of the type of things they can pick up, then have a watch of some of the short UX tips videos I have created.
It’s not an heuristic evaluation
People often confuse expert reviews with heuristic evaluations. They aren’t the same thing. In an heuristic evaluation, a non-expert can use a set of heuristics to guide them in trying spot potential issues with their designs. An expert review relies on the expertise of the reviewer, which is one of the serious shortcomings we’ll speak about later.
The two approaches have been confused to the point where some people now just use the term heuristic evaluation for an expert review because it uses big words and sounds more sophisticated. But the heuristic evaluation is a technique to be used when you don’t have access to an expert.
What makes them an expert?
The expert carrying out the review needs to have the experience of having watched lot of user behaviour. Mostly this will have come through usability tests, because the UX review will centre on usability weaknesses in the design.
If the reviewer hasn’t spent a lot of time observing user behaviour, then they don’t have the expertise required to do the review.
The problems with expert reviews
An expert UX review isn’t UX research. It is opinion-based and the whole point of UX is to design with evidence over opinion. But in terms of outcomes, the projects I’m involved in where I review designs early, tend to get better results and get results more quickly.
But the approach is not without its issues.
Lack of expertise
Back when UX reviews were more popular, they were often being carried out by people who had little experience of observing user behaviour. Expert reviews were often given by designers rather than people who observed behaviour for an occupation. While some people do both, the design side of their job dominates the research side of it massively.
When you spend your time designing things, then you grow expertise in creating designs. When you spend your time observing people trying to use designs, then you grow an expertise in how people use things. An expert UX review requires someone with the latter form of expertise rather than the former.
These days even UX researchers try to pass on usability testing to product teams and designers. Talking to users doesn’t build the necessary experience required to review designs, observing their behaviour does.
Truth be told, there appear to be few people building up the necessary understanding required to do an expert UX review. This is currently the greatest weakness of the technique currently. There aren’t sufficient people to carry them out, due to the way UX has evolved.
Bad UX reviews tend to talk more about ‘best practice’ (whatever that is) rather than the principles behind the observations being made.
They can take too long
It doesn’t take a long time for an expert to look through a design and spot the weaknesses. It does however, take a long time to report those things in writing. Writing UX review reports is slow, which is why I try to avoid that bit if I can.
I either have regular one-to-one reviews where I review work-in-progress with the designer present. This takes about an hour maximum. Or I will record an unedited video for them to watch at their convenience. This takes up about 4 hours of my time in total. A written report will take days of my time to produce.
Done in isolation
Expert reviews can provide quick results, but without there being user research in the project, the gains will be limited.
It’s a quick, first iteration approach to a design process which should have user research at its centre. I’m sometimes asked to do repeated reviews on designs which aren’t being tested with users and I often can’t confidently do so.
It’s not a replacement for a user-centred process. Instead it enriches it.
Judging impact is difficult
Even for an expert, it’s often difficult to judge the true impact of any usability issues spotted. It’s probably more accurate to say that the expert is spotting design weaknesses rather than issues.
This is why they are best done as early in the process as possible. When a weakness is live in the product, the argument for revisiting it is a lot harder to make than when it is first being proposed.
The benefits of expert reviews
On the understanding that the review is undertaken by a usability expert, then UX reviews have a number of things in their favour.
Improve faster
You can improve your designs at a faster rate, when expert reviews are a part of the process.
If I carry out usability testing of a design which I haven’t reviewed first, then a number of the issues I observe in testing will be things I would have picked up in a review. These issues block the discovery of further usability issues in the usability study.
It’s therefore more effective to remove the detectable issues at the earliest point. You can then detect issues in your first round of testing which would otherwise only come to light in the second round.
Spot weaknesses which won’t show in testing
An expert with a wide experience, can draw upon more past experience than just usability testing. Some design issues don’t lend themselves well to being spotted in usability testing. Sometimes for example, they are small points of friction which apply to numerous users and therefore have a measurable impact.
The input of a UX expert can help spot things which won’t necessarily be obvious in a usability study.
The learnings of past research
Related to the previous point, there are some patterns of issues which emerge regularly enough across a number of usability studies that you know them to be of concern. But in a single study, they can appear like a one-off issue. An expert has the benefit of observing hundreds of usability sessions and other research to pull upon. You can benefit from this by getting their input on your designs.
Improve the ability of designers
When I review a designers work, I try to explain the principles at play (as I understand them). I believe this helps in not only improving that design, but also the designer’s future work.
However this is largely dependent on the outlook of the designer. Some designers are actively looking for things to improve and others are defensively guarding their ideas.
There are all sort of reasons for the latter outlook, but it doesn’t lend itself well to getting the benefit from expert reviews, or usability testing for that matter.
Design crits are not a replacement
Recently the design crit has emerged as a technique within design teams. This is where designers share their work-in-progress and take (hopefully constructive) criticism of it from their peers. I’ve attended lots of these sessions and can see they have great value.
However they aren’t good for spotting UX issues. Whenever I have been in one, the weaknesses I can see are rarely mentioned by the designers’ peers.
This comes back to the fact that the role of a designers involves so many things that observing users is at best, a semi-regular occurrence. Usually it’s rare. The person best placed to spot UX issues in a design, is the person who spends their time studying how people use things.
That was a long plug David
Conveniently, I offer expert UX reviews. I really enjoy carrying them out and am always keen to do more. If you are interested in having me review your work then get in touch for a chat.
