You know you need to conduct research when there’s a hypothesis you’re trying to validate or a new product idea you want feedback on.
But if you want to be truly user-driven, and you’re only focusing on these projects, you may be missing critical user insights.
Continuous UX measurement helps fill this gap with “always on” surveys that measure and monitor the user experience. Often used in-product, in the context of natural product usage, these surveys capture quantitative sentiment about your product experience as well as qualitative insights about what users find difficult or want to improve (and the best part: qualitative insights at the scale of a quantitative survey)
For product managers, continuous insights turn into fast indicators on feature prioritization and value-driven optimization. Meanwhile, user researchers are like kids in a candy shop revealing untapped data on the customer journey.
Continuous UX measurement can help all companies, from successful startups all the way to established brands keep their eye on the user-driven ball. Here’s how they do it – and why continuous research is a complement, not a substitute, for one-off surveys.
Three Benefits of Continuous UX Measurement:
- Identify new opportunities to prioritize the product roadmap
- Know what to measure next
- Aligns the whole company around the customer
Continuous Measurement v. Ad Hoc Research
Let’s say a product manager wonders, “Why is adoption not meeting our benchmarks?” Or the Head of Growth asks, “Why is our conversion rate so low?”
That’s your cue. It’s time to launch a one-off or ad hoc survey.
Ad hoc research gets at specific questions that are being posed by the business to address concerns, pain points, and opportunities. You can quickly learn exactly why users aren’t adopting a new feature by asking targeted questions regarding ease of use, awareness, etc., or understand what barriers are standing in the way of users converting during the onboarding process. It’s user-focused, but ultimately driven by the business’ needs.
On the other hand, continuous measurement is focused on the user’s needs first and foremost. Continuous measurement wants to understand the user’s perception of their experience, what they think could be improved, and what would make them happier. It’s proactively pulling what you want from the user rather than pushing feature ideas on them to see what they think.
Said another way: Continuous measurement gives users the mic, while ad hoc follows the lead of business hypotheses based on metrics and assumptions.
And one shouldn’t exist without the other. The targeted questions answered through ad hoc research improve business outcomes, but you need proactive continuous measurement to make sure you’re not missing any critical insights.
Digging Deeper Into Continuous UX Measurement
Ad hoc research may be tied directly to business outcomes but continuous UX measurement is the most effective way to consistently and efficiently optimize your customer experience.
Why is that?
Because you want to get ahead of the problem. And when you are continuously surveying and monitoring your user experience, you’ll know when there are issues in the customer journey before it ever shows up in the metrics. You’ll understand whether customers are happy — or unhappy — with new features, if their experience is getting better with each release, and whether there are pain points that haven’t trickled down to revenue loss yet.
How Continuous Measurement Can Help Researchers Gain Buy-In Across Teams
When utilizing continuous UX measurement, you’ll know whether there’s substantial negative or positive sentiment developing among users, if certain features are causing hesitation, or if you have a ton of bugs and glitches you didn’t even know about. All of these things can tip your team off to future issues with churn and retention.
And because this data is quantifiable with large sample sizes, product managers, designers, founders, and others will most likely trust the research insights and act on them. TL;DR? Continuous UX measurement makes the entire organization more user-centric.
A Real-World Continuous UX Measurement Example
At Sprig, we’re using continuous measurement to ensure the user experience is getting better with each launch and product update, as well as to proactively identify pain points throughout the customer journey. We do this by asking the right questions of the right people at the right time, without disrupting the customer experience.
We ask both close- and open-ended questions, and give users the opportunity to share more, or make themselves available for additional follow-up research.
Here’s how we set it up: After 30 days, we ask both free and paid customers for feedback on their experience with Sprig. The questions are placed on one of the five highest traffic pages, after a customer has had at least five sessions, and two page views in that session and three seconds on the page. We don’t want to disrupt the flow of their experience, but we do want to catch them while they’re using the product.
We ask four simple questions so that the mental load is low for the user. But don’t be fooled, even though the questions are short, they are not easy to craft. Make sure to leverage survey-writing best practices and ensure you’re not biasing results. Here’s what we ask:
The 3 Benefits of Continuous UX Measurement
We’ve learned so much just from this one continuous survey at Sprig. Many organizations, especially larger companies with multiple product areas and more complex journeys can leverage several “always on” surveys throughout the experience.
Here are some of our major takeaways and what you can expect if you start using continuous, in-product UX measurement.
1. Identify new opportunities to help prioritize the product roadmap: At Sprig, continuous UX measurement showed us that customers weren’t confident their in-product surveys were running correctly. While we had done a lot of research with our customers, this was not something we had heard before and users said it was holding them back from using Sprig. After we dug in further, we learned that launching surveys within a product can be scary, and while users were excited, they wanted more diagnostics and previews so they could be confident when launching in-product. This was a huge insight that could have massive business impact.
2. Continuous measurement helps you know what to measure next: Answers to your in-product surveys provide ideas for the next survey, the next question. Even your research itself becomes customer-led.
3. User-driven research helps rally the entire company around the customer: With continuous in-product surveys, everyone, whether they’re a product manager or a designer or even the founder, can log in and see survey results in real time. Continuous measurement means product managers aren’t waiting months for results. Instead, they have access to in-the-moment sentiment, behavior, and adoption.
With these benefits in mind, here’s an example of how customer-centric change happens. The research teams at a large file-sharing company and a well-known fintech launched continuous measurement and started regularly reviewing in-product survey metrics. The ability to point to key, real-time metrics won over stakeholders, up through the c-suite, and rallied everyone around improving the customer experience.
Why Continuous UX Measurement Matters Now More Than Ever
Continuous UX measurement leads to becoming user-driven, and becoming user-driven leads to product and company success. When you conduct continuous research on your customer experience, you are looking proactively and holistically at things like adoption, engagement, retention, and task success to make product and feature prioritization decisions and proactively get ahead of new opportunities.
This is more critical than ever as industries face slow downs and companies need to separate themselves from the pack. With continuous UX measurement, user research and product teams can rally the entire organization behind the customer and understand how critical the customer journey is to the success of the business. And decisions need to be driven by what the customer wants — not only what stakeholders think they want.