Qualitative vs. Quantitative UX Research Methods: A Comprehensive Guide

User experience research might be a relatively new profession, but the work behind the role has always been important to the development of new products and services. From companies asking customers for feedback to developers observing how their products are used to designers relying on their own experiences and intuition in order to inform their creative decisions, data from user testing and understanding of human behavior have shaped how things are made, bought, sold, and shared.

Because of the importance of this work, user experience researchers play an increasingly important role in organizations, helping stakeholders understand user behavior, addressing usability problems, and assisting in the design process of new products or services.

Trained UX researchers have a variety of tools at their disposal to help take the guesswork out of UX design. Their methods generally fall into one of two categories:

• Qualitative research, which helps them understand user attitudes and behaviors; and
• Quantitative research, which helps them understand the statistical significance behind experiences

One way to think of qualitative research is that it answers the “why?”—why are users struggling to complete tasks? Why does the number of users fall off after a certain time? Why is the conversion rate so low?

In comparison, qualitative research helps in answering the question “How much and how many?”—it’s based on math, web analytics, and big data, and is primarily focused on measuring the size and scale of a problem.

Most UX researchers are capable of working with both qualitative and quantitative research methods, although it’s not unusual to specialize in one or the other.

Related Read: What Does a UX Designer Do?

What Is Qualitative UX Research?

Qualitative research, also referred to as “qual,” is the process of collecting and analyzing non-numerical data to understand user attitudes and motivations, and developing an in-depth understanding of a problem. Using qualitative research methods, UX researchers try to answer questions such as, “why?” and “how?”

Qualitative research generates insight into how users experience a product or service, and researchers will often use tools such as open-ended customer surveys, focus groups, interviews, and observational field studies.

Unlike quantitative UX research, qualitative research doesn’t require as many respondents because, instead of focusing on statistical significance, it seeks to understand the experiences of users and formulate theories or hypotheses that can then be tested.

What Is Quantitative UX Research?

Quantitative research, also referred to as “quant,” is the process of collecting and analyzing numerical data to identify patterns and averages, measuring data points, and producing results that, as the name suggests, are quantifiable. Using quantitative data, UX researchers answer questions such as “how many?”, “how much?”, and “how often?”

Quantitative research provides hard data, and researchers working with quantitative methods often use resources like Google Analytics to track information such as user clicks, completion rates, and conversion rates, time spent on a platform, and the results from A/B testing, etc.

In order for this type of research to be effective, it requires a large number of research participants so that enough data points can be generated and conclusions can be drawn about the statistical significance of the findings.

Qualitative vs. Quantitative UX Research Methods: Which Should You Choose When?

Both qualitative and quantitative user research methods have their own strengths and weaknesses, and it’s important to choose the right method in order to maximize relevant data collection and glean the most useful qualitative and quantitative insights.

Qualitative user research methods are best used to discover problems, investigate what is causing those problems, and find ways to fix them. Research methods such as focus groups, interviews, open-ended surveys, and observations can be done with as few as eight research participants and can help researchers identify problems that might not be evident from quantitative studies and big data. For example, an interview can shed light on why a particular web feature isn’t as popular as the design team thought it would be; an open-ended survey or observational study can explain why users stop short of checking out on an e-commerce platform, and small focus groups can tell researchers the reasons for a high or low click-through rate.

Quantitative user research methods are best used to measure usability metrics. Research methods such as analytics, A/B testing, quantitative surveys, and quantitative usability testing can help researchers determine the priority or scale of a problem. For example, where a qualitative interview might reveal that a few people don’t click through to sign-up for service because they find the navigation inaccessible, quantitative research helps determine what proportion of users are affected by a particular problem and whether it’s widespread enough to warrant a design overhaul. Quant research is ideal for benchmarking and drawing comparisons, helping companies differentiate between trends and outliers.

How Is Data Collected in Qualitative and Quantitative UX Research Methods?

User experience researchers will often employ both qualitative and quantitative research methods in order to provide the most accurate assessment possible of how users are experiencing a product or service. Below are some of the common research methods used.

Quantitative methods

1. Technical analysis. One of the most common types of quantitative user research is the process of diving into user analytics to see whether any problems show up in the data. For example, Google Analytics collects troves of data ranging from when people visit a website to the time spent on a site, pages visited, referral details, task times, browser type, scroll depth, and errors, when users try to fill out a form. From this information, UX researchers can quickly identify trends and pinpoint where a problem might exist.

2. Mouse tracking analysis. Mouse tracking follows a user’s movement on a website. With enough research participants, this methodology can be used to generate a heat map that shows parts of a website where there is lots of activity and parts that get ignored. This is useful in identifying basic usability issues, such as whether users are mistaking certain graphics for clickable buttons, or whether they are trying to enter text into non-existent fields. It can also help researchers determine scroll depth—understanding how far most users scroll can help them figure out where they need to place the most important information.

3. Funnel analysis. Similar to technical analysis, funnel analysis focuses on how many users complete a workflow and drop-off percentages for each stage of the funnel. For example, when applied to an e-commerce platform, the funnel may include steps such as visiting the shopfront, browsing, adding items to a virtual shopping cart, filling out a check-out page, and completing an order. Funnel analysis allows UX researchers to zero in on where users might be falling off, which can lead to more specific analysis.

4. User testing. This method, which invites real users to offer feedback, can be used to collect both qualitative and quantitative data. In the latter, a researcher might record a user as they perform certain tasks, measuring how long it takes for them to complete a task and noting where a user runs into problems. The quantitative data collected can be used for benchmarking and determining whether a redesign is necessary, or whether recent changes made are an improvement.

Qualitative methods

1. User interviews. One-on-one interviews or focus groups with users are an effective way of gathering qualitative data that sheds light on user attitudes and behaviors. Where quantitative methods might pinpoint that a problem exists in the user funnel, speaking directly with users can explain the nature of that problem—are shoppers stopping short of checking out because of a glitch? Does the shipping cost cause them to change their mind? Is there something about the check-out page’s design that disrupts their shopping experience? Interviews give researchers the opportunity to go deep into the user experience.

2. User surveys. Surveys can be used in both qualitative and quantitative user research depending on the nature of the questions. Open-ended surveys are often used in qualitative research because the questions tend to be broad and invite users to go into detail about their experiences. In this sense, they are similar to user interviews and focus groups, with the added benefit of researchers being able to send them to a large number of people.

3. User testing. Where quant user testing might focus on how long it takes users to complete tasks, qualitative user testing asks users to narrate their experience as they navigate a website, platform, or app, offering their thoughts about how difficult it is to complete a task and how they perceive the website (e.g. Are they trying to click on a graphic because they think it’s a URL? Are they abandoning their shopping cart because of load times? Do they keep canceling a transaction because they are accidentally interacting with pop-ups?)

4. Diary studies. Diary studies track user behaviors over a period of time and allow researchers to collect data even when they are not actively interviewing or recording a user. Some of the upsides of diary studies are that they minimize the bias that comes with a researcher looking over a participant’s shoulder, they allow the user to participate in the study in their own time, and they provide data points collected from a longer period of time. Possible use cases of diary studies include having participants track each time they use an app to order take-out—what was their motivation for using Seamless or Grubhub instead of calling the restaurant directly? Why did they decide to get takeout instead of dining in or doing an in-restaurant pick-up? Participants might also track every time they order running shoes from a retailer—was the purchase inspired by a seasonal change? Does it align with a new year’s resolution? Is it intended as a gift? Understanding these kinds of user motivation can help a business better serve its customers.

4 Common Mistakes In Qualitative and Quantitative UX Research

User experience research can play a critical role in guiding the development of products and services toward success. But the wrong methodology, a biased researcher, or lack of data can also lead design and development teams astray. Some of the more common mistakes include:

1. Not having enough information. Quantitative UX research requires a large number of participants in order to generate enough data to show an accurate, statistical significance. If there are too few participants, it runs the risk of skewing research results. A similar idea applies to qualitative UX research—although fewer participants are required, it’s important for researchers to not assume that the few participants interviewed represent all users because that can also skew research findings and recommendations.

2. Asking leading questions. While researchers do their best to be as unbiased as possible, sometimes bias can sneak into a study through the form of leading questions. For example, if a qualitative UX researcher begins with a hypothesis that an e-commerce platform’s conversion is low because the site’s design is confusing, they might make the mistake of asking participants “What do you find confusing about this page?”, which would prompt the respondent to think negatively about the page. This kind of biased questioning can lead to researchers getting a result that simply reinforces their hypothesis, without actually addressing the real problem.

3. Using the wrong method. There’s a time and place for every kind of UX research methodology, but choosing the wrong one can lead to unusable results. For example, if an app user agrees to complete a survey on their phone, it’s better to keep the survey short and sweet because a person responding on a mobile device probably doesn’t want to spend too much time answering questions. Saddling a respondent with too many questions could result in user frustration, which could prompt the participant to input random responses just to get it over with.

4. Asking the user what they want. There’s an episode of The Simpsons where Homer is given free rein to design his dream car. When the carmaker asks him what he wants, he lists every feature under the sun. The result is The Homer—a disastrous invention that bankrupts the fictional automaker. UX researchers can run into a similar risk when they ask users what they want. Instead, researchers should come in with a hypothesis that they can test with participants, keeping their research as focused as possible.

Qualitative vs. Quantitative Research Methods: Which Is Better?

Instead of thinking of qualitative and quantitative research as being in competition with each other, it’s more accurate to think of them as serving different purposes. For example, if an organization wants to better understand the motivations behind user behaviors and attitudes, which in turn drive time spent on a page, devices used, referral links clicked, or user drop-offs, qualitative user research methods such as interviews and open-ended surveys are more likely to yield in-depth insights.

Meanwhile, if an organization wants to quantify the user experience and measure load times, conversion rates, average time spent on pages, the kind of devices or browsers in use, referral links, or user drop-offs, quantitative user research methods such as analytics and user testing are more conducive to getting useful and relevant results.

Why Use Both Qualitative and Quantitative Research Methods?

It’s not unusual for UX researchers to use both qualitative and quantitative research methods because they often complement each other. Quantitative research might reveal that a certain design becomes unusable at a certain point, but qualitative research is then required to drill into the kinds of problems users encountered. Quantitative research can show that a particular problem isn’t a random outlier and is statistically significant enough to warrant fixing, but qualitative research will shed light on what changes should be made to improve the user experience.

“It is always good to use both qualitative and quantitative,” according to UX researcher, Dominic Rogers. “By using both, you can identify [a] hypothesis and then find measures and metrics that can then prove that hypothesis.”

This post was written by Tracey Lien.

Since you’re here…
Interested in a career in UX design? Rise to the top of the CV pile when you enroll in our UX Bootcamp—you’ll get a UX job or your tuition money back. Take a look at our student reviews and test out our free UX course to get a feel for our style and results. TL;DR: average starting salaries for our students = $85,440. Let’s do this.