The Most Illuminating User-Focused Activity

Have you ever heard of businesses investing millions of dollars into misguided product decisions? Chasing solutions based on intuition, unreliable feedback, and questionable data? Pouring tons of resources into feel-good design exercises and assumption-based journey mapping? How about workshops guided mostly by user lip service and “storytelling” rather than facts and data about potentially uncomfortable realities?

The Case for UX Testing

Merely talking about users, drawing inferences from analytics and surveys, or rallying product teams around Post-it Note activities isn’t real UX. Poking around user perspectives and experiences without directly observing them isn’t exactly the most trustworthy of data sources. You’ve probably witnessed it. I’ve definitely witnessed it. These situations happen far too often, given what we now know about product development best practices, driven largely by the meteoric rise of UX as a discipline.

Facing the truth can be tough. When staring down issues such as new competition, declining market share, evolving customer behaviors, and the fear of missing out on new tech innovations, it’s easier to just talk about the problem space and hide behind malleable measures of reality than to face it head-on. Fortunately for the bold, there’s a supremely efficient method of illuminating reality that’s been around for thousands of years: empiricism.

Related Read: What is UX Design?

Does Your Product Team Embrace Reality and Learn From It?

In our context, empiricism posits that observation is the guiding principle of the scientific method. Empiricism has driven the trajectory of knowledge throughout human history, from the systematic trials and errors of Thomas Edison’s inventions to what we’re learning about our universe every day. In other words, it’s served us quite well.

So why would otherwise rational business professionals forgo empirical approaches in favor of the feel-good activities mentioned above? We’ll save the ponderings of their motives for another article. Instead, let’s discuss what is arguably the most illuminating user-focused activity that product teams can undertake: UX testing.

(Note that course corrections, product changes, and new features should always be as informed as possible from a variety of sources. UX testing is not the only answer; it is but a powerful tool among many, enabling product teams to operate from a position of more certainty than they have without it.)

First, let’s acknowledge that product teams basically have two primary goals: to build new products and optimize existing ones. In pursuit of these goals, they focus on reducing risk and increasing their likelihood of success. Ideally, they make decisions and track performance through KPIs, OKRs, and metrics of all kinds, which are undeniably useful to the business if carefully chosen, properly collected, and accurately analyzed, but they rarely tell the complete story.

Those types of numbers represent reality, but the people looking at them are often far removed from actual product usage and struggles. The numbers are mostly communicated through screens and there’s a whole lot of user experiences happening behind them.

For a variety of reasons, too many teams keep their users at arm’s length, barely interacting with them, often as if they are reluctant to learn the reality of what may be revealed. Often the hardest questions to face are the ones most worth answering.

• What pain points are users struggling with and why?
• What frustrates them?
• What confuses them?
• How can we make the UX more efficient and effective?
• Where are they least confident in the experience and why?
• What do they like and dislike most?
• What should be fixed or improved upon and why?
• What do they wish the product would do that it cannot today?
• How easy to use is the product vs. the competition?

Thoroughly addressing questions like these are what the most successful product teams on the planet do. They are the ones behind the products enjoyed by you and pretty much everyone you know on a daily basis. They consistently leverage a variety of user-focused activities and invest heavily in UX testing. They employ a variety of qualitative and quantitative studies to fully understand potential frictions between user and product.

What are the rest doing? Well, it’s easier to hide behind metrics shown on computer screens, occasionally emerging from digital utopia to pat each other on the back in celebratory bouts of collaboration, than it is to commit to serious UX testing. It’s human nature.

Related Read: What Does a UX Designer Do?

What Is UX Testing?

“UX testing is a research method characterized by the observation of users while they interact with a product, system, or service (or a prototype thereof). To product teams, UX testing is the great antidote to the poison of groupthink, misguided enthusiasm, and superstition.”

—The UXology Group, with inspiration from Adam Smith (philosopher and economist)

You may know this as usability testing or user testing, but neither of those terms is accurate. UX testing is for evaluating more than just usability—and it’s not really a test of users, but of the experience itself. It’s one of the original and most proven methods in product development. UX tests, ranging from simple to complex, may consist of any of these attributes:

• Qualitative (small sample size, focused on the why)
• Quantitative (high sample size, focused on the numbers)
• Moderated (facilitated in real time by a person)
• Unmoderated (recorded by automation)

How Does It Compare?

As a method, UX testing accommodates nearly any type of product development scenario. For physical, digital, and mixed experiences, waterfall, agile, hybrids, and more, I’ve never seen a system that UX testing couldn’t fit into. Furthermore, it is harder to game than most data because what you see is what you get.

True user behaviors can’t elude observation like they can with self-reported data and limited intuition. Think of it as the WYSIWYG of research methods; trustworthiness is high because virtually no abstraction is involved. Plus, among user-focused activities, it is probably the best at getting everyone to focus on the only thing that really matters: how people respond to what you are designing and building.

To really take the guesswork out of design decisions related to product team goals, it’s wise to utilize the power of observing small samples of experiences over time. That’s exactly what UX testing is for:

• Sampling experiences through exploratory or focused observations to uncover answers to the tough use-related questions that otherwise prove elusive.
• Seeing and hearing users struggle with a prototype or product makes for compelling evidence for change.
• Observing patterns of “user errors” and pain points through targeted sampling indicates that the user population at large is likely experiencing problems too.

How Should It Work?

1. Throughout the design process, build your best-competing design ideas into prototypes and iteratively improve upon them through a series of practical UX tests, where each test/redesign sprint results in a better version to then test/redesign again. Do this repeatedly over a period of months until you get to your final, validated version, which you know will work well for people because you’ve done your UX due diligence. This enables you to consistently course-correct so you end up launching something that has already been vetted by target users, instead of relying on assumption-based design and riskily hoping everything turns out OK.

2. In this process, the best experience was determined through diligent, scientifically-driven design. Key UX issues were identified and solved early and often, while optimal design solutions were explored and validated. Now your team can confidently launch the product… but your work isn’t done yet!

3. Unless this is the only product your business will ever build and you’ll never improve upon it (which, of course, is never the case), then it’s wise to regularly collect and track user feedback. Establish an ongoing, systematic research program to provide a steady stream of trustworthy insights that you can use to improve the experience over time, or exploit to build related products and services. As the litmus test of product team efforts, UX testing should be a central component of any research program.

This approach is repeatable and scalable to experiences focused on any group of target users, for inward- and outward-facing UX efforts. It can be layered on top of, or replace aspects of, many existing product development processes. The trick is to ensure that your data is clean and unbiased, your researchers are independent, and the right UX tests are applied in the correct ways at the appropriate time. This ensures that the product team is on track to delivering successful experiences and meeting their goals, thanks to the repeated sampling of reality afforded by empiricism.

Conclusion

• UX testing is a form of empiricism that forces product teams to face potentially uncomfortable realities so they can proactively address them early and often.
• It’s where the proverbial rubber meets the road, where the realization of the team’s creations meets users’ perspectives and behaviors in the real world.
• It’s flexible, trustworthy, and fantastic at getting everyone to focus on the only thing that really matters: how users respond to what you are designing and building.
• If seriously committed to ensuring success and reducing risk, then UX testing must be an iterative activity as part of a larger research program

Whether it consists of small-sample-size qualitative feedback, large-sample-size quantitative efforts, or something in between, UX testing is arguably the most illuminating user-focused activity that we know of today. Let’s be clear here: if you’re not UX testing, then you’re not serious about UX and you’re avoiding uncomfortable realities. What are you waiting for?

Since you’re here…
Are you a future UX designer? Enroll in our UI/UX Bootcamp and join over 10,000 students who have successfully changed careers with us. Want to get wireframing right this second? Check out our free UX design course today.