Here’s a worst case scenario. You spend months perfecting a product. Pouring resources in from the engineering and design teams. It’s exactly what you envisioned.
You drum up excitement around the launch. You go to market.
And it flops.
How did this happen?! How did all that time and energy go to waste?!
If you didn’t run concept tests on your initial concepts with users, then you may not solve a problem for them or make their lives easier — without these two factors, a product can’t survive.
One famous example is Netflix’s launch of Qwikster, which separated the streaming business from the DVD business. Operationally, separating the products made sense for Netflix, but they didn’t consider their user’s experience. Two separate accounts with two separate monthly bills made it confusing for customers. Unsurprisingly, customers hated it and Netflix stock dropped 50% after the launch. Netflix apologized and nixed Qwikster but, if they’d just run concept tests with their users beforehand, they could have saved significant time, money, trust, and reputation.
On the other hand, when you do run concept tests and invest in research during the conceptual phase, there is up to a 50% reduction on the development timeline. You avoid unnecessary iterative rounds with the engineering team, you have time to build more of the right features, and fix bugs. And you have more confidence in the product you are building and the problem it solves in the market.
Too often, people don’t test their concept and prototypes because they’re strapped for time — but testing actually saves time. Investing in concept and usability testing is like a crystal ball into the future success of your product, except it’s driven by actual user needs and feedback.
What is Concept Testing?
Concept testing allows you to get user feedback on product concepts before you begin to build. You can test with your own users or panel participants to instantly review concepts, complete recorded usability tasks, and provide feedback via text, voice, or video without downloading annoying plugins or extensions.
From the start, a concept test will tell you if you are solving a real problem, meeting a real need, or if it’s falling flat. Participants can provide feedback on early ideas or designs to offer insight into the viability of the potential solutions you’re considering, and give you input on how to iterate on those first concepts.
For example, let’s say you’re launching a food truck finder app. You’ve narrowed it down to two concepts. One gives users the ability to find food trucks near them using a Yelp-like format, providing a review-based list of top food trucks in a particular location. The other uses a map to show food trucks in the area, then users expand the listing to show ratings and place pick-up orders. In concept testing, it becomes clear that the map helps users visually conceptualize which food trucks are near them and placing the pick-up orders ahead of time is useful for busy workers looking to pick up a quick lunch. Concept two moves on to the next stage.
What is Usability Testing?
Later in the process, once you’ve narrowed down and built on the winning concept, you launch a usability test. Usability testing is exactly what it sounds like — you’re making sure your product is actually usable. Participants will complete a set of tasks to identify points of friction and surface opportunities to improve user experience.
Back to the food truck app. You use usability testing to determine whether users can actually use the product to successfully locate a food truck. Some users, however, have revealed a potential issue. The app uses a geo-locator, but sometimes users are searching for food trucks in a location they’ll be in tomorrow — not where they are right now. Luckily, the product managers now have the information ahead of time and are able to do iterations on prototypes before they launch. A zip code field is added and the product is ready to ship.
What is Message Testing?
Message testing is designed to determine whether marketing language, product messaging, and copy are resonating with customers. You could test website headlines, ad copy, creative, and more.
Let’s say your food truck app has launched and it’s steadily growing its customer base month-over-month, but now you’re focused on optimizing your marketing to reach more users. You’re working on your SEO and redesigning the home page. It’s time to test a few different headlines and messages to see which appeals and converts.
With your own users, you test two headlines/subheads. “Find Food Trucks, Fast | Place pickup orders at food trucks near you” and “Say Goodbye to Waiting in Line | Find food trucks nearby and skip the line” are tested with users. One emphasizes the main functionality — finding food trucks. The other focuses on a common annoyance — waiting in the food truck line. The two headlines are tested and, as it turns out, what resonated with users was skipping the line.
Unmoderated v. Moderated Testing
While moderated tests rely on a researcher’s time to walk participants through the concept, Sprig allows participants to independently complete surveys, record audio and video, and record screen actions as they test. There are benefits to moderated research— you can dig deeper into questions and redirect lost participants, but the time and effort is substantial, and the sample size is limited.
Unmoderated testing tools like Sprig help save time by eliminating the need for a researcher or product manager to sit in on multiple individual user testing sessions and enables testing with a larger overall group.
Sprig’s in-product surveys enable you to quickly recruit participants in-app and funnel those users to unmoderated concept and usability tests to complete on their own time.
Who do you test with?
You’ve probably noticed we mentioned “your own users” a few times already. That’s because we believe that, in many cases, the best people to test with are those who actually know your product.
Obviously if this is a pre-product launch, rather than a new feature launch, then you don’t have users. Paid testers are the solution and that’s fine. They can even be helpful if you want the perspective of someone totally new to your product. Current users could have a positivity bias towards the way things currently are, whereas non-users can provide insights into obstacles that could prevent new users from becoming customers.
Paid testers also make sense if you work for a Big Tech company and everyone is basically one of your users. For example, if you’re Google, any group of paid testers probably knows how to use your products — nearly everyone is a user. But using paid testers can be risky if you have a niche audience and insights may not be as reliable.
Benefits to testing with your own users
Testing with your own users is not only less hassle for you, it can give you confidence in your insights. It’s easier for the users because they know your product and understand why and how to use it — there isn’t a learning curve or the process of explaining hypothetical situations. They’re invested in using the product successfully.
How to recruit users
You can use third-party panels through platforms like UserInterviews, email recruiting, or in-product recruiting to find the right users for your concept test. When recruiting in-product, Sprig uses behavioral targeting for a direct line to the users you care about most and prompts them to opt into your study. Sprig also gives you the ability to add screener questions before you send them into concept testing.
To increase your odds of opt-ins, avoid interrupting key flows during the ask and keep the prompt friendly, short, and sweet. Give them a reason to opt in — even if it’s just a sneak peek! People like feeling like insiders, especially if you tell them why you chose them.
Best practices for testing with your own users
Time it right, especially if recruiting in-product! You want to ask users while they’re still excited about the product, but not so early that they’ve been disrupted.
Make it simple and seamless to complete. These users are helping you. Don’t make their lives harder.
Give everyone access to view insights. By giving everyone across product, engineering, design, and other relevant teams access to these insights, it can lead to more brainstorming, more interpretations, more ideas, and more insights. Testing in a vacuum isn’t helpful — data is only useful if it moves beyond the research team.
Test with Your Own Users for Insights You Can Trust
Sprig’s in-product surveys combined with concept and usability testing lets you test quickly with your own users. When your users know your product, you get the best insights about what’s working or not working, and what to build next.
Sprig’s research platform enables teams to learn from users across the entire product development lifecycle. Start using our in-product surveys, in-product recruiting prompts, and concept and usability tests by signing up for free or scheduling a time with our team.