Bad Research: Why Any Research, at Any Cost, Is Not Better Than None
Yes, “bad research” is a thing. And its repercussions are more severe than you might imagine. In this article, we will explore how a lack of explicit intent can lead to detrimental consequences in our approach to researching and designing for our fellow humans.
What Prompted this Discussion?
My motivation to delve into this topic came from a recurring claim that I find deeply concerning:
“Any research is better than no research!”
Don’t get me wrong; this statement isn’t inherently flawed. Often, it’s intended to encourage UX students to take some steps in understanding their users, rather than completely forgoing research when faced with tight budgets and timelines. In fact, secondary research can be surprisingly insightful and inspirational.
However, the problem arises when this statement is treated as gospel, unquestionable, and when any research, regardless of its quality, is considered desirable.
In the past, I, too, championed this message and invested a considerable amount of effort and time convincing teams to engage in any form of research. Yet, in recent years I found myself more in a situation where advocating against (nonsense) research became part of my role in design.
The Importance of “Intent” in Design Research
A couple weeks ago, I wrote about how to prioritise design research, which led me to reflect on times when I should have said “no” to research. The heart of the matter is that many organisations equate being human-centred with merely checking “a research box” on their list. Being truly human-centered is so much more, and it all comes down to intent.
Take a moment to scrutinise your intent. While this may not be a simple task, it’s not overly complicated either. Your intent to do research of design should always be your guiding light.
“Any” is Not Better Than “None” if It’s “Bad”
What do I mean by “bad” research? It’s research that lacks intent. Often, it’s conducted “just because” for various reasons, such as:
- Someone told you to do it.
- It’s a mandatory part of your UX or design process.
- You have no intention of using the insights you gather, or you don’t know how to apply them.
- Your approach to design research is solely method-driven; surveys are a popular choice here.
- You’re trying to sway a decision that’s already been made, and you’re doing it alone.
- You’re repeatedly conducting the same research without any impact on the product or design.
- Being human-centric is confused with other customer-facing activities in the organisation, like sales and customer service, which have different goals and approaches (that do not take into consideration biases etc).
I must admit that I’ve fallen into the trap of scenarios 5 and 6 in the past. For instance, I have found myself in situations where product teams were desperate for more understanding of user behaviour, but they had no intention of making the tough decisions they need to make (because building what users need was actually technically difficult, yet no one expressed this explicitly so that a conversation on “what could we do now” could have followed).
So, What Happens When You Conduct “Bad” Research?
- You end up with a heap of potentially erroneous data, which can result in misguided decisions and wasted time and resources during product development.
- It yields non-actionable insights because it fails to answer the actual questions you are wrestling with or lacks any intent.
- This, in turn, leads to disillusionment with research.
- To be blunt – it leads to bad design.
In summary, the risk of “bad” research lies in the squandering of time, money, and the overall perception of design research turning sour among those involved. It’s not uncommon to encounter sentiments like,
“We did some research, but it wasn’t very helpful.”
or
“We wasted 6 months in design and development, and the adoption of what we released is not what we expected (based on research we did).”
Convincing people to give design research another chance and regain their trust can be quite challenging, and it’s crucial to delve into what went wrong and why people feel this way — often, one of the aforementioned reasons is to blame.
Beyond the lack of clear intent, another detrimental aspect I’ve observed in the execution of research is the lack of analysis or haphazard analysis methods. This gives us another topic to return to later.
