User research: unmoderated vs moderated interviews
User research: unmoderated vs moderated interviews
Is there a superior research model?
I recently read a post about the different styles of user interviews, declaring unmoderated interviews as the clear winner for user research, almost the holy grail to give user researchers a place at the decision-making table.
Many thoughts went through my head (one of them was, “This is a typical awareness campaign for someone who uses the title of a ‘specialist for unmoderated testing,’ so I don’t want to participate in a commenting game”), triggering this article.
Moderated vs. unmoderated user testing is not a new topic for discussion, but I want to shed some light on user testing for anyone new to this field.
What is user research good for anyway?
User research studies people using a specific product or service (= “users”) to understand their behaviour, needs, motivations and barriers or pain points. It’s an essential tool for product and service design. By observing and talking to the people you are designing for, you can uncover areas for improvement or opportunities for new solutions. It makes a lot of sense: if you don’t understand the problem, how would you ever be able to come up with a good solution?
Market research vs. user research
We can research people on different levels. And this is where the distinction between market research and user research is essential.
- Market research focuses on the bigger picture, the market. Its goal is to describe people as groups by defining quantifiable similarities and differences (like gender, age range, living location, and preferred brands).
- User research goes a level deeper. It looks at specific behaviour and emotional reactions for a defined situation with a specific goal or outcome in mind.
Qualitative vs quantitative data
These two different approaches result in different outcomes.
If we want to describe a whole market, we need lots of data. Market research uses quantitative methods like surveys to gain clear statistics and numbers. Numbers are great for business; they are measurable and concrete, and they seem to describe reality the best based on the large quantity.
While numbers are great and valuable, they can only capture certain aspects of “reality”. This is where user research comes in. It often (but not only) uses qualitative methods, like interviews or observations. Qualitative data is descriptive and expressed in words rather than numbers. It is usually based on a smaller group of people but also aims to find patterns (similarities and differences).
User research results are often dismissed because of the lack of quantity — but they are essential for giving quantitative data meaning. Quantitative data can measure the frequency of a certain behaviour (i.e. how many people come to a website or drop off during a checkout process). But it can’t give you the reasons behind the behaviour. To understand the “why”, we need to observe and talk to people to understand their goals, mindset, and behaviour.
What are user research methods?
Ok, so if the goal of user research is to understand the “Why”, how do we get there?
User research offers a wide range of methods, more than just interviews (the most commonly used method).
User research methods can be categorized based on four main factors:
- Research focus: attitudinal (what people say) vs. behavioural (what people do)
- Type of data: quantitative (indirect input) vs. qualitative (direct input)
- Context of study: (near-)natural setting (observing people in their environment) vs. scripted/ limited (observing people in a staged setting/ interviewing people to report about a situation)
- Research phase: generating (discovering new areas and understanding a topic) vs evaluating (validating hypothesis)
This leads to a variety of research methods with different purposes and uses.
Moderated vs. unmoderated interviews
This finally brings us to the topic of this article: What’s the fuzz about moderated and unmoderated interviews?
Moderated interviews
A moderated interview probably comes immediately to mind when hearing the word “interview”: it’s a structured conversation between two or more people. An interview has a clear focus or topic, and the interviewer has a guide with prepared questions. However, an interview is not just a conversational form of a questionnaire. It’s more like a detective game: each answer delivers another clue, and the interviewer can either drill deeper or steer in a different direction.
The real art of moderated interviews is to be able to combine observations with questions and, therefore, to “dance the dance.”
👍 Pros of moderated interviews
- Real-time insights: You get instant feedback and can ask follow-up questions.
- Deeper understanding: You can dig deeper into participants’ thoughts, uncovering hidden gems of information.
- Adjustable: You can react immediately to your participant’s response and explore new areas you might not have thought of before.
👎 Cons of moderated interviews
- Time-consuming: Moderated tests can take more time due to scheduling and facilitation.
- Observer effect: Participants might feel a bit like they’re on stage, affecting their behaviour.
👉 When to use moderated interviews
Moderated interviews are great for complex tasks or when you want to explore the emotional rollercoaster that is user experience. In addition to observing a behaviour based on a specific task, you can drill down to understand the underlying motives and needs.
Unmoderated interviews
Unmoderated interviews are more like a solo adventure: you prepare some questions and tasks in writing. The participant is then asked to “self-conduct” an interview (without the participation of an interviewer) by answering these questions and fulfilling tasks while verbalizing thoughts. There are many platforms to support unmoderated usability tests that allow participants to record themselves while having access to the questions and instructions in a guided manner.
👍 Pros of unmoderated interviews
- Time-efficient: Because you don’t have the participation of an interviewer, you can conduct multiple interviews simultaneously. Participants can take the test whenever suits them.
- Cost-effective: It’s a budget-friendly option to do testing as there is no facilitator involvement.
👎 Cons of unmoderated interviews
- Lack of interaction: You miss out on the live-action and can’t ask follow-up questions, just like watching a movie without the director’s commentary.
- Limited insights: Participants might not articulate their thoughts as thoroughly without the guiding hand of a facilitator.
👉 When to unmoderated interviews
Unmoderated tests are your go-to for simple tasks, routine evaluations, or when you need to gather a large amount of data quickly — think of them as the fast-food option for usability testing. They are especially useful if you work in an agile setting with regular rounds of testing, where each test has a specific focus. They also work well for uncovering usability issues (where a user gets stuck in a flow of interactions).
Takeaways
I hope you gained a better understanding of the differences of user research methods.
The main insights I would like you to take away are:
- User research is a helpful addition to market research
- Qualitative data is not less valuable than quantitative data; it answers the “why & how to fix it” behind the “how many & how much”
- User research is more than just interviews
- There is no one superior research method — it all depends on your desired outcomes
When it comes to picking the right method, don’t just focus on your budget and your timeline — but be really clear about your desired outcome. In many situations, it actually makes sense to mix multiple methods. You could, for example, start with some moderated interviews to discover the biggest pain points and then back it up with more unmoderated interviews to collect more quantitative data.
Additional resources
If you want to learn more about user research methods, here are some great resources:
NNG: When to Use Which User-Experience Research Methods
Maze: What is UX Research, Why it Matters, and Key Methods
User Research: The UX Research Resource Guide
User Interviews: Free Research Templates