Metrics-based research: HEART framework
A user-centered approach to influence product and business.
Constant improvement and growth of the product and its features is an important part of the product's life cycle. It is vital for us, as Product Designers, to understand how the product is performing and how our users interact with it in order to identify pain points and opportunities for improvement. Of course, we can measure and track how different features in the product are being used by analytics tools. We also can interview the users or spread surveys in order to gather qualitative data. But for more strategic, organized ‘insights mining’ a metrics-driven framework will become a useful tool in your arsenal. In this article, I will talk about how we experimented with Google’s HEART framework at my company, focusing on the process of working with qualitative data.
HEART framework comes from Google and was developed by its Lead Designer Kerry Rodden. Its purpose was to help Google’s UX design teams narrow their focus to only a few key metrics to evaluate the quality of the user experience and help businesses by evolving a UX strategy.
My company provides a broad range of solutions for HR and the goal my team had is to find a framework we can roll out across all the products. To try HEART framework we decided to initially focus our attention on one product at a time.
Preparation
First of all, before diving directly into action, we did some research. First of all, we needed to learn more about the framework and tried to find any case studies of how exactly it was used by other companies. However, no detailed process was usually described, hence there still were a lot of questions to be answered. Where to store the data? How to combine different types of data? How to monitor further? How to analyze the qualitative data?
One of the challenges we faced was the complexity of our product and its scope. That is why, to get a better understanding of any crucial pain points, we decided to narrow down the scope of the research to one main flow.
Miro is the tool my team uses on a daily basis, therefore all the research and collaboration was done on the miro board. There was even a template for HEART. We mapped out all the metrics we would measure, as well as goals and signs.
The next step was to identify which tools and methodologies we should be using to get the full picture.
Happiness — means user’s satisfaction with the product and requires qualitative data. To measure Happiness the simple survey will do a good job. You can consider incorporating SUS, HATs frameworks in HEART. But as they usually consist of several questions, we did not want to overwhelm our users. For the survey, we used a tool called Pendo. We created a short survey with one rating question and one open-ended question.
Engagement, Adoption, Retention, Task success usually require quantitative data and can be measured by tools like Google Analytics, Heap or Quicksight, or in product analytics if your service/website is built in WordPress, Wix, Instapages, etc. However, you still can get relevant insights from the survey you run to measure Happiness. Depending on what exactly you are measuring, you might need more than one tool. In our case, we partnered with data analysts to retrieve necessary information from Quicksight.
Action time!
For the survey, we selected 9 clients and defined a 2 weeks time frame. The same timeframe was set to collect quantitative data from Heap and Quicksight. However, as we were getting a lot of responses, we decided to close the survey couple of days earlier.
In total, we had 2386 poll responses and 501 comments for the open-ended question. To retrieve the data from Pendo, we just downloaded the spreadsheet and review it in Excel. However, with 501 written responses it seemed pretty inconvenient.
I never knew, but Excel works magically with Miro. The Product researcher I was working with easily transferred the comments into Miro's post-its with the client names, and the rating they gave displayed as tags. Genious.
Then we grouped them all into clusters according to their rating score. Of course, there were a lot of positive things shared, but this exercise really helped us to identify things our users are not happy about. And doesn’t matter which rating the user gave, they still mentioned the same things in different score groups.
This really helped us to understand the major pain points in the investigated flow and prompted us to set the next steps beyond the HEART framework. The comments users left, also became the points of research within every metrics we were looking into. These comments helped us understand some rationale behind the results of engagement, adoption, retention, and task success.
During this phase, we stored both qualitative and quantitative data findings in the miro board to have it all in one place. This way is great to present the analysis and discuss within the team. To keep the quantitative analysis going, we created dashboards inside the analytics apps we were using.
What’s next?
After all the initial steps and arrangements, the further process of measuring will be easier, but in need of constant tracking and maintenance. Having the templates for a survey, analytical reports, and dashboards means they can be duplicated and reused by other teams. The ultimate goal is to roll HEART framework out as a common practice across the company, to keep the focus on our users.
Thank you so much for reading, I hope these notes are helpful. Looking forward to any feedback and comments. Cheers!
