A Framework for Demonstrating Research Impact

Measuring research impact is crucial for generating positive change within your organization. But how can you do it consistently, with reliable results?

The answer: a framework and a process.

Brandie Smith, currently Lead User Researcher at WayBetter and previously Senior Principal User Researcher at Metromile spent her entire career—including in the role of User Researcher at Facebook—designing surveys and studies focused on user experiences and behaviors, conducting qualitative and quantitative research, solving complex problems, and delivering research roadmaps.

Smith recently sat down with us for an episode of our People Driven Products podcast. In her interview, she introduced us to a research framework she uses for demonstrating impact. Brandie originally learned about this framework from Victoria Sosik, Director of UX Research at Verizon, who defines research impact as “when the knowledge generated by UX research influences another person, organization, product, or strategy. Her process can help solo and team researchers develop a consistent methodology that delivers real results for organizations as a whole.

What is Research Impact and Why Does Tracking it Matter?

Victoria and Brandie both believe that it's always the influence your work has, not just the work itself, that really matters.

As a user researcher, tracking and measuring the impact of your work will help you:

Better prioritize and socialize your research
Understand whether to invest in more (or different) research methods
Know which research project has the most impact on the product roadmap
Identify how (and whether it’s possible) to replicate the success of your most impactful research projects
Gain confidence in your research and leverage it for your career goals

This, in turn, can help demonstrate your team’s impact on the organization as a whole. Increasing the credibility of your research team can make the case for additional investment—and you may also be invited to help make strategic decisions about the future of your product or service.

Demonstrate your team’s impact on the organization

We made a framework template for you to use to consistently and reliably quantify your impact.

Download the framework

Why is it Hard for Teams to Measure Research Impact?

In her long experience as a UX researcher, Brandie Smith explained that she’s noticed the following common barriers to research impact measurement:

Time Limits

Especially if you’re a solo researcher or a small team, limited time can be one of the biggest barriers to research impact measurement.

Research teams have busy schedules and often need to wrap up a current project in order to focus on the next biggest priority—which is usually another big question to answer. Time limits make it difficult to document and properly contextualize research results.

Time Between Research and Application

It’s fairly easy to document the impact of evaluative and tactical research (like concept or usability testing), particularly when it informs a change in the product or experience before it’s shipped.

But for generative or foundational research, it might take months or years to see an impact on the product. Given the wider time gap between the initial research and the analysis of the results, it’s much easier to let documentation slip through the cracks.

Inability to Tie All Outcomes to Research

Sometimes, experience improvements are made in less structured ways.

Discussions, brainstorming sessions, and even short workshops that rely on research can lead to positive product and service changes. But these different collaboration types make it hard to determine what drives every change—and in turn, accurately measure the impact of research on outcomes.

Lack of a Framework or Process

Many research teams falter because they simply don’t know where to begin. They don’t have a solid understanding of what to measure or how to demonstrate impact. That’s why a well-grounded framework and process can help teams conduct reliable UX research and measure results predictably.

A Framework for Measuring Research Impact

The inspiration for Brandie Smith’s framework comes from a talk given at the online UXR Conference Anywhere 2021 by Victoria Sosik. At a high level, Sosik’s framework has three main parts:

The research activity that drove the impact. Often, this will be a specific research study or a cross-functional workshop.
The impact or the recordable instance of influence. Sosik tracks eight different types of impact, including influenced product change, influenced product strategy, and stakeholder participation.
The scale of the impact. How you define and categorize the scale of impact depends on the size of your company. Impact scale runs the spectrum from individual stakeholders to the organization and beyond.

Using this framework, Brandie Smith tracked her own impact at Metromile, and now at WayBetter in a Google sheet using the following process:

1. Note the Impact

Write a descriptive note of the observed impact, along with the date. For example, an entry by Smith might read: “I Led Leadership through a 10-minute tagging exercise of NPS comments. CEO slacked me, ‘this is such a smart way to engage leadership.’”

2. Document the Research Activity

Record the name and date of the research activity—such as a study, a workshop, or a document—that this impact came from. The key here is that it’s not just a deliverable but an activity that encompasses all stages of research as a whole.

In Smith’s case, she leads a monthly meeting where she and a cross-functional team compiles a holistic view of the customer experience, which is then presented to leadership. For this, Smith might write down that the research activity was presenting "Product's NPS Tagging Exercise during the Monthly Customer Experience Leadership Meeting.”

3. Determine the Focus Area Affected

Make a note of the main research focus area affected by this impact. This is personalized to your organization and might include:

Lanes of work (eg. Mobile experience, customer experience)
Teams/squads (eg. core product, growth product)
Product areas (eg. Enrollment, claims)

Using the example above, Smith would note that the impact relates to Metromile’s customer experience lane of work.

Demonstrate your team’s impact on the organization

We made a framework template for you to use to consistently and reliably quantify your impact.

Download the framework

4. Note the Type of Research Activity

Sosik’s original template included evaluative research, generative research, and iterative research. In her impact tracker, Smith adds one additional category: “company initiatives.” This distinction allows her to track the impact of internal research as a separate category from product research.

For the example above, Smith would mark her entry as an iterative research activity, since it stems from a monthly meeting she leads. She considers presenting NPS during the monthly Customer Experience Leadership meeting to be an iterative research activity because "it involves a systematic repetition of a sequence of tasks executed in exactly the same manner multiple times, provides a deepening understanding of research data and brings a standard of reliability to the research."

5. Select the Impact Type

Then, decide the type of impact that was observed. Categories in Sosik’s framework include:

Influencing a product change
Influencing a product strategy
Increasing stakeholder exposure to users
Sharing communications
Prompting further research
Prompting a new collaboration
Elevating the role of user research
Developing infrastructure

Smith also added “mentoring” as an impact type to capture the work she does in supporting the research activities or skills of others.

In our example, Smith could categorize this impact as relating to increasing stakeholder exposure to users, or elevating the role of user research.

Keep in mind that receiving praise usually isn’t considered impactful, because researchers should look for the actions that happen after receiving the positive feedback (such as product changes or business outcomes). In this case, since Smith received praise from a harder-to-reach senior executive, she considered it as elevating UX research status.

Praise may be worth documenting if it helps to build the credibility of research when you and your team are just starting out. But more established teams would do better to focus on product and product strategy impact. In order of scope, the best types of impact to look for include:

A usability issue identified in a prototype test is corrected before launch
The project manager mentions insight from research in a meeting about product strategy
The CEO mentions insight from research in an All-Hands meeting
Insight about users leads to changes to roadmap
Insight about users leads to changes to product strategy

6. Determine the Scale of Impact

Lastly, note the scale of the impact observed. Smith’s choices include:

Individual stakeholders
Individual project or team
A larger body of work
Organization/company
Beyond the company (like the broader UX community)

For the example above, Smith would select organization because it was a group of people rather than a body of work.

Pro Tips for Tracking Impact

Include links to important documents. Link to roadmaps, strategy documents, Sprig study results, or experimentation details outlining your research. Having easy access to these documents lets you reference them later for more context.
Note specific teams that used your research. That way, you can provide helpful follow-ups and keep ideas circulating.
Modify the framework tracker to suit your needs. Smith’s framework provides a comprehensive starting point, but it might be missing something that’s important to you or your organization. For example, if you have a career leveling document, look at the types of impact listed under each level and see if that variety can be captured in the framework. If not, find a way to incorporate it.
Update your impact tracker often. Make regular time for it (even with something as small as a 10-minute block on your calendar every week) so it doesn’t get pushed to the back burner. It might also be helpful to make this practice a part of regular check-ins with teams and stakeholders, since you won’t always be involved in every decision-making conversation.
Don’t start from scratch. Download our Impact Tracker excel workbook to start tracking your impact without having to build the document yourself.

Wrapping Up: Use This Framework to Measure Your Research Impact in 2022

When thinking about research impact, many UX researchers only consider the impact on the organization’s product or strategy. But there are also intangible interactions that demonstrate the research is truly having an impact on the organization, like being invited to strategic discussions or obtaining resourcing.

Using Brandie Smith’s framework for tracking research impact can help you prioritize research, replicate successful results, and increase your team’s credibility in your organization.

It can also give you and your team the confidence you need to keep going. As Brandie Smith puts it, “Any time I feel imposter syndrome creeping up, I take a look at my impact tracker and remind myself that I’m actually doing a lot—and having a significant impact.”

Of course, one key to measuring the impact of your research is having enough time for big-picture analysis. You’ll need to use the proper tools, templates, and systems for gathering data as efficiently as possible. Luckily, tools like Sprig can help you obtain in-context insights from your users as they experience your product, so you can demonstrate the ongoing impact of both your quick tactical projects and larger strategic research initiatives. Schedule a call with our team to learn more.

Get the best content on user insights, design, and product management delivered to your inbox every week.

Written by