5 Things You Might Be Getting Wrong About Prototypes

Testing is an important part of product development, but you might be testing the wrong thing, in the wrong way, for the wrong reasons. Here’s how to design your tests for maximum impact.

Photo by Amélie Mourichon on Unsplash

“I have observed organizations invest huge time and resources to create extensive UX prototypes. Alternatively, developing a small working increment is quicker.” — Biplab Subedi

I did, of course, point out that they were Doing It Wrong. A UX prototype should never require huge time or resources, or be slower to build than developing the code.

More to the point, there seemed to be a mismatch between the thing being tested and the thing they were trying to find out.

Let’s go back a little.

1. You might be asking the wrong question

Forget, for a minute, about “UX research” and “market research” or any specific role, and take on the mantle of a product strategist. To develop a successful strategy, as business guru Roger Martin puts it, you have to ask, “What would have to be true to make this a winning idea?”

Product development becomes a set of questions, leading to a set of choices. You ask the question, find the answer, make the choice.

The successful product owner will enlist others to help them answer those questions in the form of experiments, including market researchers, UX researchers, QA, analysts, salespeople and helpdesk workers.

• What is the market size and value?
• What do people in this market need?
• How many people have this need?
• How much would they be willing to pay to solve that need?
• How are they solving it right now? What are the weaknesses with that solution?
• What can we uniquely and profitably provide that will be better?
• Is this, (conceptually) in principle, better than the current solution?
• Do the tasks that this solution performs solve the problem?
• Can the user understand the information on the page? Can they successfully navigate a user flow?
• Does the technological solution work?
• Does the user know how to use it?
• Does the user enjoy using it?
• Where can this be presented to them in the most attractive way?
• What messaging and materials will persuade them that this is better than the alternative?
• What do they complain about? How can we remedy those issues?
• How can we increase adoptions and reduce churn?

If the only questions you’re asking are “do they know how to use it?” and “how can we increase adoptions?”, you’re Doing It Wrong.

2. You might have the wrong person asking the question

Many of the problems in the product space stem from having someone design and deliver research who is unsuitably trained.

Untrained researchers can deliver biased and misleading results, which can result in failed products.

“Would you buy this product?”

“Uh … sure, I’m sure I could find a use for it.”

(Dear reader, they will not buy this product.)

In my list above, the first five questions should be answered by market researchers and/or UX researchers. (In many companies, the role is combined into a single Customer Insights function.) The “does it work?” question can be answered by QA. The complaints question can be handled by the helpdesk.

Make sure that whoever is managing the experiment has the appropriate qualifications and experience to deliver reliable results.

3. You might be using the wrong methodology

A Figma prototype cannot tell you if the product works, in a technical sense. A code build on a plain landing page cannot tell you if the user will be able to navigate the finished app.

Let’s go back to my questions list:

• Is this, (conceptually) in principle, better than the current solution? (You can test this with a mocked-up advert without building anything at all)
• Do the tasks that this solution performs solve the problem? (You can manually perform the tasks yourself that will later be automated)
• Can the user understand the information on the page? Can they successfully navigate a user flow? (You can test this with a low-fidelity interactive wireframe in Figma)
• Does the technological solution work? (You can test this with a section of working code on a plain web page — the functionality without the design)
• Does the user know how to use it? (You can test this, unmoderated, with a high-fidelity interactive mockup in Figma and a set of written tasks)
• Does the user enjoy using it? (Get 5 users on a Zoom call, get them to share their screen and talk you through their experience of using it)

Choosing the right methods means you don’t have to do more work than you need to in order to answer the question.

4. You might be too attached to the idea

One of the advantages of adopting an experimental mindset is that you fixate on the problem, not a particular solution.

If you know that 92% of all products fail, you will want to get as many ideas into the pipeline as possible, and eliminate as many of them as you can before spending too much time or money.

If you can eliminate an idea with the mocked-up-advert concept test — “Oh, I don’t think I’d use anything like that at all!” — then that’s wonderful news! You can put all that time and effort that you would have spent developing that into something else that customers will love.

Think of product ideas as contestants in a game show — almost all will be eliminated. Save your time and money for the winners.

5. You might be building too much

The term “minimum viable product” gets a bad rap, for a good reason. It is neither supposed to be so scrappy that nobody would ever buy the thing, nor so full-featured that it’s a bloated product before it even hits the stands.

The term “vertical slice” is more easily understood. It’s a video game term, describing the demo level that is shown to reporters before the game’s release.

Think of a game like Fallout 3: the demo level (or opening scene) takes place in a nuclear bunker, so its scope is contained. You can’t go everywhere and do everything that you will be able to do in the finished game, but you can create a character, level up skills, interact with NPCs and objects, experience the storytelling, and generally try out most of the core functions of the game.

That’s the mindset with which most MVPs should be built: pick just two or three core user flows and build only enough for the user to complete those journeys, but with the richness of quality that will make them hanker for more.

Harnessing a ‘game demo’ mindset can help you focus on only the most important elements, so you can create a beautiful experience without bloating the product with features it might not even need.

In summary

1. Think like a strategist; frame your tests as experiments
2. Get a researcher to help you research
3. Match the method to the question
4. Murder your darlings — weed out the weak ideas
5. Don’t overbuild!

Happy prototyping!