Our InterQ blog often features topics focused on specific qualitative methodologies, such as in-depth interviews and focus groups. While InterQ helps a lot of companies with qualitative research projects, we also help companies with UX testing and UI design, which is the subject of today’s blog. Today we tackle whether you should use qualitative or quantitative methods in UX.
Anyone who designs or builds apps or software will (definitely should) first do some UX testing before launching the product to the public. Whether or not you do the UX testing internally, or outsource it, one of the decisions you’ll be confronted with is should you conduct a qualitative or quantitative test.
If you’re wondering which would provide the best results, you need to first understand the fundamental differences between the two types of testing.
A qualitative assessment would involve engaging with potential users and soliciting their feedback, whereas a quantitative test would focus more on data and metrics, such as clicks and cart abandonments.
When conducting UX testing, both quantitative and qualitative methodologies are useful and, we’d argue, necessary, but sometimes there are budget constraints or more specific questions that need to be answered, and one methodology may be more appropriate over the other. Here is a quick, and simplified, checklist to gauge whether quantitative or qualitative is the best option.
First ask this question before deciding on quant or qual in UX testing
“Do I need to know what is happening or do I need to learn why users are taking a particular action?”
The “what” question lends itself to a quantitative UX study. Your metrics should yield some helpful data on core stats, such as clicks on a specific page, web traffic, referral sources, time of day of engagement, and other stats that can help paint a picture of WHAT. Your team should be keeping close tabs, at all times, on this data. With some digging and sorting, your team should also be able to find additional quantitative data about what is happening that will paint a clearer picture of actions taken.
And now to the “why.”
Quantitative data is great when you’re sorting through lots of stats and information. You might easily be able to see that most of your clicks are happening after 5 p.m. While this is useful, it doesn’t tell you is “why,” and – even more crucially– what else? What else did the user want to accomplish but couldn’t? What information was the user seeking that couldn’t be found? Analyzing clicks and heatmapping won’t tell you WHY and what else, but qualitatively interviewing users and observation will.
Blending quant and qual for optimal UX study outcomes
In an ideal world (Ha!) product teams and designers would engage qualitative assistance for new product design and modifications. If your team has this capacity, here are some straightforward ways you can blend quant and qual so you end up with a better design for your users:
- Pay attention to your funnel metrics. Are certain issues popping up, such as users not taking the next action along the mapped-out journey?
- Use quant metrics to help pinpoint where problems/issues/abandonment are occurring and follow-up with the help of a qualitative team to observe/ask people going through these specific steps. Start with the quant (what) and finish with the qual (why) to better understand user frustration and drop-off points.
- Set up A/B tests. If you’ve followed the recommendation above, use the qualitative input to design two or more scenarios and set up A/B quant testing to see which option gets the most hits. For a well-rounded study, we suggest going back after the A/B test and interviewing people to understand their experience with both options, and why they selected one over the other. The more you seek to understand what else could be improved to further enhance the design and intended pathway, the more successful your product will be.
Last, but not least: Outsource qual to the professionals
Our last tip for exceptional quant and qual UX testing is to outsource the qualitative portion. Designing qualitative studies that aren’t biased is harder than it seems. If you’re going the extra mile to engage directly with users, make sure the information you receive is useful. Ask your data team to pull metrics, and relay this to a trusted and experienced qualitative market research firm. Partnering with a neutral research agency ensures your research isn’t tainted by the myopia and assumptions that internal stakeholders have when trying to design qualitative studies. A neutral research firm is objective and can see your product from an outsiders’ perspective, similar to your customers. From a neutral position, an outside research company can come up with interview questions that can probe the ‘why’ behind customer behavior. Bringing a fresh perspective to UX testing will help unlock new ways to design software and apps, ensuring that your end product will succeed with your customers.
