The Decision Matrix Product Managers Can Use To Launch Features People Really Want

If you stumble on Nissim Lehyani’s LinkedIn bio, you’ll notice that he wears many hats.

Since graduating college, Lehyani has co-founded two companies, worked as a developer, and served in two other companies as Director of Product Management.

As a Senior Director of Product Management at Indeed, he is helping the company adapt to the shifting needs of the 250 million job seekers who visit the platform every month. In conversation with Sprig CEO, Ryan Glasgow, Lehyani describes product management as the act of enabling a better path that reduces friction between people and a valuable outcome.

Lehyani is driven by one goal: creating frictionless products that delight users. This commitment has helped Indeed build and launch products that users love.

For example, in 2020, the team launched a video interview platform to make hiring quicker and easier in the social distancing era. At the time of the interview, more than 20,000 U.S. job seekers had been hired using this feature.

In our interview with Lehyani, he shares several pages from Indeed’s playbook that Product Managers can draw insights from.

In this piece, we'll unpack the strategies that have helped Lehyani and Indeed create products and features that stick—including the equation he uses to decide what products to build and four tips for assessing what features are the most valuable.

Let's get to it.

The Equation for Identifying What Features and Products To Build

Prioritizing product features and capabilities can be pretty tough.

It’s worse when you are faced with multiple choices and several variables and forced to identify those that will make or break your product. The pressure can be paralyzing.

One sure way to clear up confusion and weigh your options is to use a decision matrix.

A decision matrix is a table—a series of values in columns and rows that lets you visually weigh several options and make the best possible decision:

A product-specific matrix would consist of:

• A set of viable solutions—the top features users really want
• Criteria or variables that help you analyze each feature

Lehyani calls this type of matrix a “Product Maximization Equation.” At Indeed, Lehyani uses these three variables to identify ideal features:

• Amount of value to be delivered
• Size of the audience
• Length of time to ship

He uses this exact decision matrix to create frictionless products at Indeed, and he has done it at other organizations as well. When describing this methodology he emphasizes that ‘one of the best things you can do is throw a lot of ideas and stack rank them based on key variables.’

This matrix helps user researchers identify features that most users would find valuable if added to an existing product. It also helps PMs weigh these options against each other, using a set of variables to determine the most relevant feature.

Here’s how Lehyani describes the importance of this equation,

“Having the ability to prioritize and scope is crucial. You need to systematically identify the ideas that will deliver as much value as possible to the largest audience as possible, and as fast as possible.” - Tweet this

Using a matrix, PMs can eliminate sentiment and guesswork from the decision-making process and help their team better understand what’s a priority and what isn’t.

So, how does the Product Maximization Equation work?

It starts with defining the goal of your product—the problem your product solves.

Say you run a burger joint, for instance.

Your goal, of course, is to give people something delicious to snack on—a fast-food everyone would typically enjoy. However, to achieve this goal, you’ll need to constantly listen to your customers to learn about their preferences so you can better serve them.

That’s where the second step comes in:

Finding out what most of your customers consider valuable.

If you conducted some industry research, you might learn that similar burger shops are offering more variety. Perhaps, you learn from the industry that there’s an appetite for three things: burgers with pineapple, burgers with bacon, and a new 100% vegan burger. The next step would be to use data to fill in these three variables:

• Amount of value to be delivered
• Size of audience
• Length of time to ship

Using these three variables to determine the best burger option, you can lay out your options and variables on a matrix, with the former arranged on a row and the latter on a column, and assign a multiplier to weigh the variables that you view as most important:

Lehyani says that PMs should assign values based on the highest priority variables when analyzing and comparing features with one another.

For example, longer speed to market would equal 1 and the shortest possible speed would equal 10. A low number of likely users equals 1, and a high number of users equals 10. These values are assigned based on the importance of the variable.

For instance, if most people who surveyed found bacon the best option, you can assign a 9 based on their feedback. And if it takes a longer time to create and serve vegan burgers quickly, you can assign a 2 or less based on the assessed speed to market.

After analyzing user feedback, categorizing it based on those variables, and assigning values, your decision matrix would look like this:

The numbers on the “total value” column show that people are more interested in the bacon option than any other choice on the list. Bacon would make a burger more valuable in the eyes of customers.

However, in terms of audience size, the vegan option stands out as it opens the door to an entirely new market of customers that weren’t always being served at your restaurant. At the same time, the existing customers who enjoy beef burgers can still try and enjoy the vegan option. Finally, when we look at the speed to market, the vegan burger falls short as it will take a lot of R&D, while the pineapple and bacon burger lead the way.

When you add up the scores for each option, you'll observe that bacon has the highest total score (23). It's most valuable because the majority of people you surveyed found the option valuable and it can be implemented in a short timeframe. In other words, choosing this option means that you can deliver optimum value as quickly as possible to the largest segment of your target market.

At this point, you might be feeling a little hungry, but we hope you get the point we’re trying to make: you can choose your next product feature using this formula.

Let’s see how Indeed’s product team used this model to build a high-growth product that connects job seekers and the right employers.

Deconstructing Lehyani’s Path For Frictionless Product Design

In our chat with Lehyani, he talked about how Indeed used the Product Maximization Equation to determine that the Virtual Interview Platform would be a feature that users would find the most valuable amidst the pandemic.

The team reached this conclusion using feedback from their users and applying the data to the same framework as our imaginary burger joint. Lehyani described feedback as being a guiding factor to frictionless product design by saying that:

Incorporating user feedback has to be in your decision-making process for product decisions and it has to be continuous. How do you capture it? You can talk to people who use the projects. You can ask them within the product. Or you can measure their behavior in the product.

To identify the problems users were facing, the Indeed team reached out to users to gather feedback firsthand. They found the #1 challenge that users faced were interviews.

People could still apply for jobs, but interviewing was a problem. Restrictions and health risks made securing jobs almost impossible. Employers could no longer hold physical interviews, and job seekers couldn’t commute miles for physical interviews.

The world was officially remote, and most companies weren’t prepared.

Lehyani and his team had other potential features they could build for their audience. However, using the maximization equation, they found that the Virtual Interview Platform was what job seekers around the world valued the most during the pandemic. Indeed considered the speed to delivery, size of the audience, and perceived value to the audience and determined that this was the feature that would maximize value.

Here's a hypothetical example of what a matrix for these three features would look like using the Product Maximization Equation after conducting research on the relative value of the product, the possible number of users served, and potential speed to market:

The data represented isn’t real, of course. However, it shows how the Indeed team would have applied this equation to identify the best-fit solution for the moment.

Let’s break down the Product Maximization Equation variables.

Here are the steps to understand each variable in your situation:

Step 1: Measuring the Amount of Value to Be Delivered

It’s important to measure how valuable different potential product features would be to existing and potential users. That way, you can identify opportunities to make your product more useful, increase traction, increase customer success, reduce churn, and increase your customers’ lifetime value.

Here are four tips for assessing the value of new features:

Let’s explore each briefly.

Tip 1: Keep it open-ended (when possible)

When discussing the importance of collecting raw and unfiltered feedback, Lehyani emphasizes that he's "a big fan of blank text boxes.”

He explained that using blank text boxes when asking for feedback allows product teams to get unadulterated feedback—raw, genuine thoughts that aren't tainted or predefined by a product team's own beliefs, biases, or assumptions.

Most PMs avoid using this feedback method because sorting through tons of feedback can be boring and time-consuming, especially if the product has scaled. It is almost impossible for one PM to go through hundreds of thousands of words to identify relevant patterns from user feedback and group them accurately.

Luckily, AI has made it easier for PMs to analyze each user’s feedback to understand what features your users would find most valuable.

Solutions like Sprig’s Survey Response AI Analysis review each survey response, identify relevant patterns and deliver them to you. All of the answers are summarized into an organized to-do list, and Sprig will flag the most relevant insights you should prioritize.

Here are 8 sample questions that your product team could adjust and use inside of your product with a blank text box:

• What was your experience like doing XYZ?
• Is there anything preventing you from buying at this point?
• What feature do you think is missing from [Product Name]?
• How has your experience been with [Product] for the last X months?
• What is the main thing you want to do with our product?
• What feature is most valuable to you?
  

When you ask a question like, “What was your experience like doing XYZ?” you’re giving users that chance to say, “here's what I thought about XYZ...” They get to share what is on their mind, and you can uncover the features that excite the most people or identify unmet needs.

Tip 2: Share concepts and prototypes with your users pre-launch

If someone is already using your product, they are invested in ensuring it succeeds so they can keep using it. Sharing concepts and prototypes with your users pre-launch is an excellent way to identify product weaknesses before new features are released to the world and to identify a ‘value score’ that can be included in your matrix.

Modern-day concept & prototype testing is a PM dream. You no longer need to get all of your users in a room and in front of a computer screen with your prototype. You also don’t need to manually book a 1-1 Zoom call. Thanks to solutions like Sprig you can show users designs and mock-ups with embedded Figma prototypes along with video recordings explaining the concept in more detail and providing directions for how to interact with it.

As users go through this prototype demo, you can obtain feedback through open-ended questions via text or video responses (users are more likely to leave a long, detailed response using video vs. spending time to type it out). Then you can quantify the rich qualitative feedback you receive to identify a score for your product maximization matrix. Our Measure Product Value template which asks users to rank the product’s value on a scale of 1-5 is a great starting point for gathering this type of information.

These unmoderated studies automate the mundane task of walking users through your prototypes when all the questions you will ask are pretty standard and free you up to work on other higher priority tasks that need your attention. And as a bonus, you get more responses and therefore have more confidence in your data.

Tip 3: Track their behavior in real-time

A common way to gain insight into what users find valuable is tracking behavior in real-time and targeting them with experiments at critical moments in their journey to gain feedback. This ranges from continuously observing how customers use a new feature, running an A/B test, tracking how they interact with a new button, or simply engaging with a new feature that you’re showing to a small group of users. Running these data-driven experiments helps teams better understand what features resonate most with them.

For example, the Indeed team could have introduced a “conduct your interview online” button throughout their product to gauge user interest before launching the virtual interview platform. This is a common practice amongst many startups to conduct A/B tests amongst a small portion of their users to see how they respond. Facebook and Amazon are both well known for running multiple experiments at once and letting the data guide whether a feature is considered and developed for mass usage.

Analytics software like Amplitude, Heap, and even Google Analytics would come in handy in this case. The number of clicks on a button or new feature would offer insight to the analytics team to measure whether or not the majority of new and existing users would find a feature useful. When a visitor or user clicks the button, they could be notified that the feature wasn’t available yet, but they would be notified once it was.

The potential for behavior tracking doesn’t end with testing potential features. Teams can also conduct controlled, randomized experiments directly in the product experience with the goal of learning what users would find most valuable. In this case, the experiment would ideally be a low-lift design experiment rather than a complete product or feature launch to reduce the amount of effort required. These in-product experiment tools help you measure the impact of your experiments by comparing how different experiment groups behave with different features.

For example, you can show two potential features to similar groups, then use the results to determine which one is more valuable in the eyes of your users. This form of product testing is a great way to get a sense of product value and fill out your matrix.

Lehyani cautions PMs using behavioral data exclusively by expressing that both behavioral data and qualitative data should be a part of your puzzle. He states:

“Behavioral data is very immediate - people complete or don’t complete an action. It’s very fast and offers lots of confidence. But it doesn't always tell you the entire story. When I go to the DMV, I convert every time (by renewing my license or doing similar activities), but it doesn’t mean I enjoy the DMV’s product. Looking at the behavioral data with the qualitative data is very important.”

Which takes us to our fourth tip for measuring product value...

Make it easier for your users to give you quantitative feedback using in-product surveys. We talked about the value of ‘open-ended’ questions earlier but simple variable-based surveys can be a lot less intimidating for users and deliver a ton of value.

Most users are unwilling to commit the time needed to fill out lengthy surveys. Customers want the feedback process to be quick, easy, and convenient. This is where short surveys become the perfect solution.

Responses to simple questions like: ‘How valuable would you find [Solution]?’ or a ranking question like ‘Which of these features would be most valuable to you?’ can help a product team identify the value associated with the product ideas they’re toying with.

Lehyani and his team use in-product surveys to find out what features and products their customers would find the most valuable. For example, to assess the value of a feature you are planning to bring to the market, you can use this template from Sprig’s research template library.

STEP 2: Measuring the Size of the Audience

The next step is to identify how many people will benefit from using the feature. Lehyani says that the size of the audience is important. You want to deliver the maximum amount of value to the largest group of people.

PMs can use total addressable market figures to ignite interest and gain approval from executives. If your addressable market isn’t large enough for significant growth, it will be difficult to rationalize investing resources in that direction.

So how do you measure and determine the size of the audience interested in a feature?

First, gather intel. You want to collect two types of data:

• Quantitative data: Do you have clear numbers outlining total market share for a new feature? Have you conducted research amongst your existing users? Are there third party reports validating a new or existing market? Is there data that exists surrounding the market within other industries?
• Qualitative data: Have some of your users communicated that they want something different or have an interest in trying a new feature? Is there a new competitor or feature in the industry that is generating buzz? Are the forums in your community filled with dialog about a certain problem?

You can gather the data using the methods we detailed in step one— surveys, concept tests, and real-time user behavior. You can also track industry trends and forecasts to discover new audiences for your product. These critical insights will help you make more accurate estimations on the number of people interested in a particular feature.

Then, analyze the data to determine the feature with the largest number of people that can be served (addressable market). As an example, if you’re planning to introduce an integration in your product with two different solutions that are very similar - you can run a short survey asking existing customers how many of them would use product A vs. product B. The total volume of customers suggesting they would use one product vs the next can be leveraged as a datapoint surrounding the total addressable market.

STEP 3: Estimate the Time to Ship

After measuring the size of the audience that will benefit from your potential new features, you need to also estimate and compare the time it would take to build and ship each potential feature.

You can estimate the amount of work a team member can do within the shortest time possible. The time is measured in weeks and months, specifically in terms of "person-weeks" or "person-months."

For instance, if it takes four team members to develop a specific feature within a week, then you can estimate that the project will be completed and ready to be shipped in four person weeks. Another feature might require two people to complete it within a month, or two person-months. When you quantify each feature in this manner, it's easier to compare each feature option to determine which will take the least time and resources.

Alternatively, you can use dev weeks. One dev week is equal to 40 hours (eight hours a day, five days a week, one developer). So, if a feature takes two dev weeks to ship, it means it'd take 80 hours to ship if one developer is assigned to build that feature.

You could achieve the project in less time by assigning two developers to complete the project in one week. It all depends on the resources you have available.

To make accurate estimations that are well-grounded in reality, it's best to collaborate with the engineers and team leads who have the knowledge you need. A simple way to do this is to simply ask everyone on a feature or product team how much time they estimate it to build a certain feature and then take the average of those estimates to identify the value that will be placed in your decision matrix.

Here’s What You Should Do Next

When you are dealing with thousands of users daily as Indeed does, you realize that the best methodology creates a scalable system and see why this prioritization framework is a great way to launch products people want.

Overall, the maximization equation lets you evaluate ideas and identify those that will deliver as much value as possible to the largest audience and as fast as possible. - Tweet this

If you are a PM looking to move faster and make better product decisions, you can leverage this equation to help you make better, quicker product decisions. Use the framework, leverage it, and you too will build products people get excited to use.

Get the user insights you need to build better products quickly and confidently with Sprig