A Primer on Mixed Methods UX Research
There are research purists — doing one type of research with great finesse. But in the applied UX setting, one is more likely to be like a mixologist, collecting information from different sources and serving the results up to their teams in a convincing package. Some analytics here, some surveys there. Perhaps a dash of interviews even.
Is that you? Do you ever wonder what EXACTLY is this method, and how do other people do it? Read on.
What is Mixed Methods Research?
Any time the researcher finds herself using both qualitative and quantitative data sources, the study is called a “mixed methods” study (not to be confused with multi-method study, which is when the study contains multiple sources of only one type).
It is a fairly broad definition coined some 30 years ago, but that’s exactly why it helps to know what prior knowledge already exists about it.
When to use Mixed Methods research
We’ll start with the underlying philosophy behind these methods.
Philosophy is important, because research question and philosophy come before any methods have been selected. ( More about this here). A researcher would use mixed methods if a purely constructivist or purely realist view is not suited to their research question, i.e when only qualitative, or only quantitative studies don’t suffice for the given research question or for the given domain.
In the literature this is called the “purpose” of the mixed method study. Creswell proposed some basic classifications for this purpose:
Triangulation: Can the results of two different studies (QUAL + QUAN) be combined to give a better picture? As an example, one could suggest improvements in hospitals by triangulating two different perspectives — a survey of patients, and a qualitative interview with doctors.
Explanation: We’ve often heard quantitative data gives a what but not a why. This solves the problem by using the results of the quantitative study as a starting point for a qualitative study. For example, one might look at the analytics of a webpage, figure out which pages have dropped traffic, and then ask customers about it in interviews.
Exploration: Qualitative studies can give a description of issues but not their statistical importance. This kind of study design solves the problem by using the results of a qualitative study to do a quantitative study. For example, to find out the most common uses for a product, ne would conduct a qualitative study to make a list of all the different uses for a product, and then run a survey to understand how common those uses were.
Each of these purposes makes up for the inherent weakness in each of the individual methods.
(There are more exhaustive lists of classifications for purpose — but this simple classification may suffice for many UX projects. Those looking for more should check out the reference links.)
How to put together a mixed method study
Let’s say one of the above purposes is suitable for your research question. What now?
Designs for Mixed Method Studies
Technically many designs are possible, but Mixed Method studies can get complicated fast due to requirements for validity. (more on that below). The following simplified designs are a good place to start.
- Triangulation:
Notice that the two studies are conducted simultaneously but separately. The two studies are combined only at the results stage, by displaying the results together. An example of this is a “joint display”.
2. Explanation:
Here the two studies are conducted sequentially — the result of the quantitative study is used to conduct a qualitative study which gives deeper insights into the numbers. This is known as combining the studies at the analysis stage.
3. Exploration:
Here again the studies are conducted sequentially — the result of the qualitative study is used to conduct a quantitative study , to quantify how common the issues are. This is again an example of combining the studies at the analysis stage.
Document the design
Creswell recommends being transparent about why a mixed method methodology is chosen, and how it has been put together.
First, write down the stages in the design and state the aims of each stage. Then clearly define the method for each stage. Lastly, show the overall design and how it will flow. This type of diagram can help:
Validity
A big issue with mixed method studies is validity. Are the methods being used in a rigorous and systematic way?
First, each of the methods has to make methodological validity on their own.
Then, the overall study must also have mixed method validity. This depends on the study design, whether the integration between methods is being done in a systematic way. For example, if a quantitative study is being followed up by a qualitative study, are all the important points from the quant study being addressed in the qual study?
These considerations will help a researcher define their mixed methods designs better, justify their use and get better insight.
Stay Curious.
Further Reading:
Designing and Conducting Mixed Methods Research, John Creswell
References:
NIH Office of Behavioral and Social Sciences. (2018). Best practices for mixed methods research in the health sciences (2nd ed). Bethesda: National Institutes of Health.
Kelle, U., Kühberger, C. and Bernhard, R. (2019) ‘How to use mixed-methods and triangulation designs: An introduction to history education research’. History Education Research Journal, 16 (1): 5–23.
Students’ persistence in a distributed doctoral program in educational leadership in higher education: A Mixed Methods Study — Nataliya V. Ivankova*, † and Sheldon L. Stick**
Conceptual Issues and Analytic Strategies in Mixed-Method Studies of Preschool Inclusion — Shouming Li, Jules. M. Marquart, Craig Zercher
