If you’re interested in capturing real-time insights into your product or service, some UX research methods may fail you.
Don’t get us wrong—there’s definitely a time and a place for methods like interviews and email surveys. But there’s also a lot to be gained from researching your customer experience while they’re using the product.
That process is in-product research in a nutshell.
Allison Dickin is an experienced user researcher who specializes in helping digital product brands make customer-informed decisions. She started in the field of market research but quickly moved into the UX field, specializing in quantitative and then mixed-methods research at Etsy. She joined our team at Sprig almost three years ago to help build out best practices around in-product research and integrate it with other research methods UX researchers use.
Dickin recently joined us for an episode of our People Driven Products podcast, where she laid out the basics of what in-product research is and how it can benefit teams. Then, she walked us through five scenarios where in-product research might be the best choice for gaining invaluable insights into your customers’ experience with your product or website.
Taking the time to implement in-product methods can help your team learn what customers want, adjust in real-time to market needs, and make informed decisions about the direction of your long-term research projects.
What is In-Product Research?
Like many UX research methods, with in-product research, you ask questions of your users while they use your product or website via a short in-product survey. The key difference with in-product research is that you're studying real users during actual usage, instead of in a scheduled research session where you ask users to simulate their typical usage.
In this case, we’ll be talking specifically about unmoderated forms of in-product research. This means using highly targeted, in-product prompts that users receive as they are using your product. The goal of these prompts may be for users to:
- Answer questions,
- Share feedback, or
- Complete a usability task.
The Benefits of In-Product Research
It’s virtually impossible to conduct pure ethnography in the world of digital products, especially since the beginning of the COVID-19 pandemic. So at least to some extent, every method besides in-product research strips away the context of natural usage.
For instance, usability testing (moderated or unmoderated) attempts to simulate the real context, but it isn't authentically in-product. This method might be fine or even preferred for some use cases, but not all.
Another common form of research is contextual inquiry, which is a moderated, in-depth form of research where the researcher visits the user and observes them in their context. The researcher's presence somewhat changes the natural context, but it is still a valuable method, depending on your goals and the part of you user journey you are focused on. Additionally, contextual inquiry isn't really possible if you're focused on the onboarding journey, for example, because you can't predict who is going to sign up or when. But for experienced users doing typical tasks, or to understand the broader context of what users do when they're not using your product, contextual inquiry can be tremendously helpful.
The biggest difference between contextual inquiry and in-product research is the size of the samples, whether the research is moderated, and how in-depth the study is. In-product research uses unmoderated methods to capture insights from larger samples on a real-time basis, while contextual inquiry focuses on a narrow sample size with an in-depth, moderated, study.
In those cases where that context is critical to understanding user needs and behavior, especially with hard to reach audiences, or users early in their journey, in-product research is really the only solution.
In-product research is one of the only ways to capture what users actually think and feel as they use your product and complete specific actions. Because questions are asked in the moment, researchers can obtain amazingly specific, deep insight—even from a quick 1-2 question study (especially when the study includes an open-ended or video question). And that’s why this method is ideal, even for qualitative researchers.
3. Understanding Why Users Are Not Trying a New Feature
It's easy to identify and learn from users who used a specific feature. But it can be harder to identify the right set of users to help you understand why users are not using a feature.
In cases like these, in-product research can reach users actively using your product (who could use this new feature, but for some reason, aren’t). For example, you could target users interacting with complementary features, who are likely good candidates for using this new feature too.
In-product research is a quick and easy way to identify a new feature’s major barriers to use. You may identify issues that can be rapidly addressed or, if the problems are more significant, explore them in greater depth through 1:1 research later.
Real-Life Example
Chipper Cash recently launched a crypto trading feature, but they didn’t see the degree of uptake they expected based on pre-launch research. They hypothesized that a lack of understanding of crypto may be driving this issue.
But after launching an in-product study with in-app, eligible users, the team found that the biggest barrier was simply a lack of awareness that the feature was available.
4. Gauging the Impact of Product Launches Pre- and Post-Launch
Does your team need a way to understand the effectiveness of product updates? Running a pre- and post-launch test in-product can produce more reliable data, with quantifiable results.
Such a test might look like asking users a few quick questions about a specific feature before rolling out improvements. Then, ask the same questions again after the rollout to gauge the impact on the experience.
Compared to a handful of live interviews, in-product research is particularly useful for gauging the user’s experience with the product because it’s done in the moment of experiencing that feature; there’s no recall required.
While interviews can uncover issues with the experience (old or new), it can be difficult to know with certainty that a revamped experience is definitely better without quantifying it. Also, having feedback from users can add important context to behavioral metrics like time on task or usage.
5. Ongoing User Experience Tracking
Finally, in-product research is ideal for measuring your users’ progress towards improving the overall experience—or the experience across specific customer journeys—over time.
Compared to some metrics programs like NPS surveys via email—which are out of context and receive much lower response rates—this always-on approach means you can surface new issues as they arise. Obtaining open-text feedback lets you detect early warning signs of emerging UX issues.
There are many other benefits to using in-product research in this way, including the ability to create customer-driven KPIs both for individual product teams and the company as a whole. This methodology is a great way to get leadership (and the whole company) more oriented around the customer experience.
Real-Life Example #1
Several of Sprig’s customers have rallied their organizations around a system of in-product survey metrics, including the research teams at a large file-sharing company and a well-known fintech. These metrics have enabled the teams to get stakeholders, up through the c-suite, engaged and focused on improving the customer experience. These metrics are a great way to shift toward a more customer-centric organizational culture.
These researchers see much higher response rates with Sprig compared to email—with the added benefit that they can see responses flow in over a longer period of time.
How Sprig Can Help You with Your In-Product Research
When it comes to UX research, the key is finding a balance between speed and depth.
In some cases, you need to conduct long-term studies to get deep results. In others, you may not have the time, resources, or executive backing for a large study.
That’s why you need a solid in-product research tool in your arsenal.
In-product research gives you valuable insights quickly so you can adjust your product in real-time, without sacrificing the more in-depth research you’re already doing.
And the added bonus? The insights gleaned from in-product research can even help direct your deeper research efforts. A quick in-product survey may generate insightful questions or hypotheses that you can dig into in your long-term research.
Ready to start using in-product research to understand the product journey, measure success of a feature, or more? Get started with Sprig today.
5 Instances Where In-product Research is the Right Method for Your Study
1. Understanding Key Moments in the User Journey
Do you want to understand why users are dropping out of the onboarding process, or a specific flow, such as the acquisition or sign-up phase?
Transitions between key stages of the customer journey are often the most important parts of the product experience from a business perspective—but they are also among the most difficult to research. This is precisely because the context is so important.
What is the user thinking at that moment? Why didn’t they click that next button?
Asking later just won’t get you the same detailed information.
It’s typically difficult to recruit research participants or recreate the experience on-demand for this flow. In addition, these users tend to be less engaged and less reachable for research.
For example, if someone simply visited your website, then navigated away, their identity would remain anonymous. You would have no idea how to contact them to ask why they left.
That’s why in-product research is likely the best tool for this scenario.
Real-Life Example #1
Nova Credit was able to identify a large gap in awareness of the company’s offerings that prevented users from signing up.
By launching an iterative series of 1-question surveys on their marketing site, they were able to uncover that while many visitors were in their exact target audience, the biggest reason they weren’t signing up was that they simply didn’t understand that Nova Credit offered the solution they were looking for.
As a result of the changes Nova Credit made after in-product research, they were able to see an approximate 50% increase in conversions.
Real-Life Example #2
In-product research helped our research team at Sprig better understand how to improve activation rates. Our 1:1 research with non-users suggested signing up for a free account was often part of the initial evaluation process, rather than a clear indicator of intent to try.
Based on this research, we launched a Sprig study that surfaced to new sign-ups asking them about their goals for their session.
We were able to validate and quantify that insight in a way that made the next steps for building on the first-time user experience very clear.
2. Measuring Success of a New Product or Feature
Hopefully, you do effective research leading up to the launch of a new product or feature. But even the most well-researched feature can perform differently—or interact with other features in unexpected ways—once it goes live.
In-product research is an excellent way to get specific, relatively representative feedback from your earliest real users, whether it’s to:
- Identify bugs,
- Surface unexpected usability issues, or
- Identify opportunities for product iteration to enhance value.
In-product research can reveal more concrete and specific details about issues or experiences customers are having when they interact with the feature for the first time.
Real-Life Example
Our customer Square’s e-commerce team launched a series of features at the start of the COVID-19 pandemic to address the needs of restaurants that serve customers via low- to no-contact methods.
Because of the state of the world at that time, they had to launch those features very quickly and with very limited research pre-launch. So they launched a series of in-product studies that triggered immediately after a user reviewed those features or completed a task with them.
The team quickly realized that the features were very important to customers. But via open-ended responses, they saw that there were a few key areas that needed additional effort to meet customer needs.
In response, the team made quick iterations—and ongoing studies reflected improvement. Their customers also expressed gratitude and relief that Square was able to rise to the challenge and help its customers keep their businesses afloat during a difficult time.