How to Run: Concept Test with Your Own Users

Introduction

Test a concept on your own users when the decision concerns current users, such as an extension of a feature they rely on, and field it in the product to target by behavior. Don’t use it to size demand or judge a new category, because in-product sampling reaches only current users and respondents can differ from those who don’t answer. For those questions, recruit people who haven’t adopted you.

Last updated October 2026.

Why the Audience Decision Comes First

Most concept testing advice starts with design: monadic or sequential, a five-point scale or an 11-point one. Those choices matter, and the parent guide to running a product concept test covers them in depth. But a design question only arises after an earlier one: who will see the concept.

Your own users need no recruitment spend, though they come filtered by who opens the product. They’re already inside the product where the idea would live, you know their plan, their tenure and what they did last week, and you can reach them in days. That convenience is exactly why teams default to them.

Practitioner advice leans the same way. Great Question’s concept testing guide says, "Whenever possible, recruit participants from your existing customer base," and adds that "Customers already understand your product, workflows, and challenges" (retrieved October 15, 2026). That’s sound for some concepts. The version retrieved doesn’t use the term selection bias, and it doesn’t say which concepts the advice fails for. This guide names the cases where it fails.

What a Concept Test on Your Own Users Is

A concept test on your own users puts a written or visual description of an idea in front of people who already use your product and measures their reaction before you build. The test itself is the same instrument the parent guide describes. What changes is the audience, the channel and the claims you can responsibly make from the result.

Three Ways to Reach Your Own Users

You can reach existing users in three ways, and each fixes a different part of the sample.

In the product: an in-product survey triggered by an event or an attribute, shown to people while they use the product.
By email: a survey sent to a list you choose, reaching people whether or not they’re active.
By link: a shared link you place in a newsletter, a community or a support reply.

In-product fielding reaches only people who open the product during your fielding window. Email reaches your list, including dormant accounts. Link surveys reach whoever sees the link, and because they’re anonymous, you can’t tie answers back to what a user did. This guide treats the in-product route as the default and names where the other two fit better.

What Stays the Same

Design, sample size, scoring and the readout don’t change because the audience is your own users. A concept test still separates concepts that are far apart and still can’t forecast demand. The parent guide’s sample-size tables and its minimum detectable difference arithmetic apply unchanged, so run that arithmetic first and use this guide for the audience and fielding decisions around it.

When to Run a Concept Test on Your Own Users

Run it when the people who would use the concept are people you already have. The condition is narrower than it sounds, and it’s worth testing each of the cases below against your concept before you pick this audience.

Extensions of Something Users Already Rely on

A new reporting view, a bulk-edit option or a redesigned onboarding step is an extension. The people who’d use it are using the product today, so their reaction speaks for the people the decision affects. Your own users also hold context an outside panel lacks: they know the workflow the concept changes, which makes comprehension easier to earn.

Decisions That Only Affect Current Users

Some decisions never touch prospects at all. Retiring a legacy feature, changing a plan’s limits or reorganizing navigation affects the people who live with the current version. Testing those concepts on outsiders measures the wrong population, and your own users are the correct audience rather than a convenient one.

Concepts Where Behavior Can Sharpen the Audience

Because you know what existing users did, you can target the people a concept is for. You can filter on attributes, groups and event history, including how many times a visitor triggered an event. A concept for heavy exporters can reach people who export often, and you can compare their reaction with occasional users. An outside panel can’t offer that.

Early Screening Before a Wider Test

Your own users also make a cheap first filter. Use them to remove concepts people don’t understand or clearly don’t want, then take the survivors to an audience that hasn’t chosen you. The parent guide frames a concept test as a filter, and this is the same logic applied to audiences.

When Not to Run a Concept Test on Your Own Users

Five situations call for a different audience or a different instrument. Read the first and third before the others, because they decide whether your own users can answer the question at all.

You need to know how people who haven’t adopted you will react. In-product surveys run through an SDK installed in your product, so they reach only people who use it. LinkedIn’s survey team stated the limit plainly: "we cannot sample non-adopters to take the voyager survey."

Better instrument: a concept test fielded to a panel or another outside audience, using the design in the parent guide. Panels is Sprig’s route to an outside audience.

You need to know whether respondents represent the users you didn’t reach. Even among current users, the people who answer can differ from the people who don’t. Better instrument: a response rate check by user stratum with weighting outside Sprig, or an email survey to a probability sample drawn from your customer list.

You’re testing a new category or an unarticulated need. Christensen and Bower found in the disk drive industry that established firms' resource allocation follows their powerful existing customers, writing that "Projects targeted at technologies for which no customers yet exist languish for lack of impetus and resources."

That finding concerns investment, not survey sampling, so apply it with care. The better instrument is discovery research with non-customers, and the AI-moderated interview guide shows one way to run it.

You need to know whether anyone will act. A stated reaction to a description isn’t behavior, and the parent guide covers the evidence on that gap. Better instrument: a behavioral test with a real action attached, such as a fake door button or a waitlist, built in your product. A survey can follow up with the people who acted, though that pairing needs confirming in your own setup.

You need a price or an attribute ranking. A concept test scores the whole bundle. For price, use Gabor-Granger. For which attributes drive preference, use conjoint or MaxDiff.

Choosing the Audience

The audience fork has four branches: own users in the product, own customers by email or link, an outside panel, or a blend. Work down the questions below and stop at the first branch that fires.

Is the Concept an Extension or a New Category?

The tree below starts with one question, so classify the concept first. Ask three things of the concept, not the audience. Would someone have to change something they do in your product today? Would the concept appeal to people who don’t use you at all? If you removed your brand from the description, would current usage still predict who’d want it?

A concept that changes existing behavior and wouldn’t interest an outsider is an extension. A concept that serves a need your users don’t currently bring to you, or that outsiders would want, is a new category. Most concepts sit between the two, and for those the blended branch is the safer default.

Be wary of calling something an extension because it’s convenient to test. The label decides which claims you can make later, so write down the reasoning before you see any scores.

The Decision Tree

Does the concept change something current users do today? If yes, own users are a legitimate audience. Continue to the next question to pick the channel.

Is the concept a new category, or aimed at people who don’t use you? If yes, own users can’t answer it alone. Field an outside audience, and treat your own users as a comparison arm if the decision also touches them.

Do you need people who aren’t active this month? If yes, use email, because in-product fielding reaches only people who open the product during the window. If no, in-product fielding is the default.

Do you need to cut results by what users did, not just who they are? If yes, fielding in the product with event and attribute filters is the strongest option, because behavior is already attached to each user.

Is the decision costly or hard to reverse? If yes, run both: own users for the affected population and an outside arm for everyone else, then compare the two before anyone commits.

Comparing the Four Audiences

Each audience reaches a different group, and each carries a different main risk. The table summarizes them, and the paragraph after it states the points that matter most.

No row is the safe default. In-product fielding is the strongest on behavior and the weakest on reach, while a panel is the reverse. The blended branch exists because one audience’s weakness is usually another’s strength.

What the Fork Costs in Practice

The blended branch needs two separate studies, so plan the sample for each arm using the parent guide’s tables. Weighting and nonresponse adjustment happen in your export or a connected AI client, not in the survey tool, which later sections explain.

Sprig Note: In-product fielding uses event and attribute targeting, and email and link delivery reach people outside the product. Limitation: Sprig documents no weighting or nonresponse adjustment, so that step happens outside the platform.

The Recommendation

Use your own users for extensions and for decisions that affect only them. Add an outside arm for anything new. And check who answered before you report a score, because no audience choice fixes a sample you haven’t examined.

Design, Instrument and Sample Size Live in the Parent Guide

This guide doesn’t rebuild the design chapters. The parent guide owns the concept statement, the intent and probability scales, scoring, benchmarks and the readout. Read it before you build anything, and come back here for the audience and fielding decisions.

Three points from the parent guide matter more when the audience is your own users. First, write every concept to the same length and structure, so you aren’t testing copywriting. Second, put a comprehension item before the intent item, since a user who misreads the concept evaluates their confusion. Third, derive your sample size from the gap you need to detect, not from a figure copied from another page.

A fourth point is specific to this audience. Don’t mention your roadmap, a launch date or a customer commitment inside a concept statement. Your users know your company, and an implied promise changes what they’re rating. Keep concepts descriptive and equal, and don’t reuse language from your marketing.

Audience, Targeting and Screening Inside the Product

Define who’s eligible before you launch, then target the moment and the segment the concept is for. Everything later in this guide, from the bias check to the readout, depends on knowing who could have answered, not just who did.

Define the Eligible Population First

Write down who counts as eligible before you build the study. That might be active accounts on paid plans, or anyone who used a given feature in the last 90 days. Record the head count and the share by plan, tenure and activity level at the time you launch.

You can’t audit who answered unless you know who could have. A response rate by stratum needs a denominator, and rebuilding that denominator after the fact from a changed user base is slower and less reliable than saving it on launch day.

Target the Moment the Concept Is About

An event trigger puts the question in front of someone at the point the concept is most relevant. If the concept changes how people export a report, trigger it after an export. If it changes onboarding, trigger it on the step the concept would replace.

That’s a practical rule of thumb, not a sourced result, and it carries a cost. Event-triggered sampling concentrates on people who perform the event, so it excludes users who stopped doing it. Those users may be the ones a concept would help most, which is why an email arm often belongs next to an in-product one.

Target by Attribute and Event History

Attribute filters narrow eligibility by who the user is: plan, role, company size or any property you pass to the product. Event history filters narrow it by what they’ve done, including how many times, and when they first or last did it. Combine them to define a segment, such as admins on a team plan who exported at least a few times this quarter.

Be deliberate about how narrow you go. Each segment you’ll compare later is a separate comparison, and the parent guide’s sample-size arithmetic applies to each one, so a segment too thin to support a comparison shouldn’t be a reporting cut.

Segments Worth Comparing

Three cuts earn their place in most concept tests on your own users. Compare heavy users with occasional users, since people who rely on a workflow may judge changes to it differently from people who touch it rarely. Compare long-tenured users with recent ones, because newer users are less settled in the current workflow.

And compare by plan, because a concept can matter very differently to a free user and an enterprise buyer.

Decide those cuts before you see the data. A cut chosen after you’ve looked is a search for a story.

Screening Inside the Product

You rarely need to screen on facts you already hold. Plan, role and activity are known, so filter on them instead of asking. Screen only on what the product can’t see, such as whether someone does the workflow elsewhere.

Keep any screener behavioral rather than attitudinal. Asking whether someone is interested in the category selects for people who’ll score the concept highly, and the parent guide explains why that manufactures a result.

Adding the Outside Arm

When the concept is new, or the decision reaches beyond current users, field the same instrument to an audience that hasn’t adopted you. Keep the concept statements and questions identical, field both arms in the same window, and report them separately. Pooling the two arms hides the one number you ran the second arm to learn: how far apart they are.

The gap between arms is itself a comparison, so it inherits the sample-size arithmetic from the parent guide. Panels is one route to the outside arm, and an email list of leads or lapsed accounts is another.

Sprig Note: In-product studies can be triggered by an existing or new event, including a no-code event for web studies, and "If User..." filters narrow eligibility by attributes, groups, event history, and no-code event URLs. A study supports up to 10 triggers. Limitation: events from server-side integrations can’t trigger a study, though they can be used in event history, and the documentation page reviewed doesn’t say how to exclude users.

Fielding the Concept Test Inside the Product

Fielding in the product comes down to four decisions: when the survey appears, how fast responses arrive, how many concepts each person sees, and what happens when someone has already seen a study. Make each one on purpose.

Choose the Trigger and the Pace

An in-product study in ongoing mode collects responses by a daily count or by a sampling rate per day. A sampling rate lets you control what share of eligible visits see the survey, and a leaky bucket approach spreads responses over time instead of letting them arrive in a burst when traffic spikes.

Pacing matters for validity, not just for load. A fast burst over one afternoon reflects whoever was active that afternoon, while a steadier flow across several days and times covers more of your users. The LinkedIn survey team sampled continuously, rotating one user bucket each week, and noted that their in-product survey could only reach users who used the product during the survey period.

Write the Invitation Carefully

The first thing a user sees is the invitation, so keep it plain. Say roughly how long the study takes, and don’t describe the concept’s benefits before the concept statement does it, because an invitation that sells primes every answer after it. Don’t promise that the feature will ship, and don’t imply the answers will change your plan.

If you offer an incentive, decide before launch and offer the same one to every segment. An incentive offered to one group can change who responds, which is the exact problem the bias audit is trying to measure.

Keep One Concept per Respondent

The parent guide recommends a monadic design as the default: each respondent sees one concept. In the product, you’ll usually build one study per concept, with the same instrument and the same targeting.

This guide doesn’t rely on random assignment across concepts, and you shouldn’t assume it. The documentation describes rotation when several surveys share a trigger, and says response rates can differ based on filters and recontact periods. Rotation isn’t a randomized, balanced split. If your design needs random assignment, read the parent guide’s design chapter and confirm the mechanism before you launch.

Handle Repeat Exposure

A recontact waiting period sets the minimum number of days before the same user receives another study, and it applies to people who saw a survey but didn’t answer. By default, a user never sees the same study twice. A per-study option can let people retake it, so leave that off for a concept test, because a repeat respondent counts twice in a cell.

The waiting period is account-wide. If several teams run studies, one team’s survey can hold a user back from yours. Check the setting and talk to whoever else surveys your users before you pick a launch date.

Monitor the Strata While the Study Runs

Check the response count by stratum partway through fielding, not only at the end. If new users are filling far more slowly than heavy users, you can widen the window or loosen a filter while the study is still live, which is cheaper than discovering the gap in the readout.

Change only the audience controls. Don’t change a concept statement, a question or the scale mid-field, because responses collected before the change and after it are no longer one dataset. If something about the instrument is wrong, stop, fix it and restart the study.

Pick the Stimulus and the Device

A written concept fits naturally in the survey itself. A visual concept can live in a prototype delivered by shareable link, and the documentation lists linking to one from an in-product survey as a delivery option. Test the path end to end on the devices your users have, because Recorded Tasks aren’t supported on mobile.

Launch Specification

Use this table as a build sheet. Each row names a decision and what the documentation says about it. Every row is also covered in the paragraphs above, so a reader who skips the table loses nothing.

Pre-Launch Checklist

Work through these in order. Each is cheap before launch and hard to fix afterward.

Write down the eligible population, its head count and its shares by plan, tenure and activity.
Match every concept statement for length and structure, and remove brand names, prices and superlatives.
Put a comprehension item before the intent item.
Set the trigger to the moment the concept is relevant, and add the attribute and event filters.
Set the sampling rate so responses arrive across several days and times, not in one burst.
Check the recontact window with every team that surveys your users.
Turn off retakes.
Test the full path in production, because the recontact window is enforced only there and a test visitor in a development environment sees a study whenever the filters match.
Preview the survey on web and on mobile, including any prototype link.
Compute your minimum detectable difference from the parent guide and write it in the study plan.
Decide the strata you’ll check for response rate, and write them down before launch.

Sprig Note: Sprig delivers in-product surveys through a website or mobile app, spreads responses with a leaky bucket algorithm, and applies an account-wide recontact setting that can be overridden per study. Limitation: a Custom 100% sampling setting prioritizes one survey and can lower response rates for other surveys on the same trigger, and Prototype Tests are being deprecated as their functionality moves into Surveys.

Timing and Recontact

Run the test when the decision is still open and before anything is built. A test fielded after the roadmap is committed becomes a document people cite selectively, as the parent guide notes, and the value of cheap evidence disappears once engineering starts.

Leave It Live Long Enough

Keep the study running until your smallest cell clears its target, not until the total does. In-product studies fill at the rate eligible users trigger the event, which you can estimate from your own analytics before launch. Estimate it, because a concept aimed at a rare event can take far longer than a concept aimed at a daily one.

Avoid a window that spans a launch, a pricing change or a holiday. The LinkedIn paper’s comparison between two surveys was complicated by mismatched sampling frames and seasonal effects, and a concept test comparing two audiences faces the same trap when the two arms run in different weeks.

Space Studies Around the Recontact Window

If you plan several waves, such as a first round and a revised-concept retest, the recontact waiting period decides who can see the second wave. Someone who saw round one can’t receive round two until the window passes, unless you override it for the study. Keep the instrument and audience identical between waves, and decide up front whether returning respondents are in or out.

Sprig Note: The recontact waiting period is the minimum number of days Sprig waits after a user receives a study, and a 30-day setting means no new study for at least 30 days. Limitation: the per-study override applies only to web and mobile studies, link studies are exempt, and a development environment ignores the window.

Checking Who Answered

Compare the people who answered with everyone who was eligible before you report a score. This is the step most teams skip, and it’s the one that determines which claims your result can support.

Why Respondents Differ From Everyone Else

The LinkedIn survey team gave the clearest published statement of the problem for in-product surveys. They wrote that results are more vulnerable to nonresponse bias when the difference between respondents and nonrespondents is larger or the response rate is lower, and gave the bias as (one minus the response rate) times the gap between nonrespondent and respondent means.

In their case study, the overall response rate was about 0.6 percent. Members actively job seeking in the US responded at 0.96 percent against 0.16 percent for inactive members, and their NPS differed by 22.1 points. That’s one company, one metric and an arXiv paper rather than a peer-reviewed journal article, so treat the numbers as an illustration of the mechanism, not as a norm for concept scores.

The Bias Audit Worksheet

Run this after fielding, using the strata you wrote down before launch.

List the eligible population by stratum, such as heavy, occasional and new users, with each stratum’s head count.
Count the respondents in each stratum.
Divide respondents by eligible users to get a response rate per stratum.
Compare each stratum’s share of respondents with its share of the eligible population.
Compute the score per stratum, then compare the unweighted total with a weighted one.
Write one sentence for the readout saying who answered and who didn’t.

Use static attributes you hold for everyone eligible, not just for respondents. LinkedIn weighted responses by job seeking status and activity level, static attributes known for the whole population, and reported similar adjusted results from weighting and from a regression-based method. They chose weighting because it was more interpretable.

Weighting by Stratum

Weighting by stratum gives each group the influence it has in your eligible population. The weight for a stratum is its share of the eligible population divided by its share of respondents. The table below uses hypothetical figures to show the arithmetic, not real data.

| Stratum | Share of eligible users | Share of respondents | Weight | Hypothetical top-two-box | |:----------------:|:-----------------------:|:--------------------:|:------:|:------------------------:| | Heavy users | 20% | 45% | 0.44 | 52% | | Occasional users | 50% | 40% | 1.25 | 34% | | New users | 30% | 15% | 2.00 | 22% |

Unweighted, the top-two-box is 40.3 percent: 45 percent of 52, plus 40 percent of 34, plus 15 percent of 22. Weighted by eligible share, it’s 34.0 percent. The 6.3-point gap exists only because heavy users answered more often and liked the concept more.

Weighting corrects only for the attributes you chose. LinkedIn’s authors cautioned that "we do not know how much bias is left," and they’re right to. Report the weighted and unweighted numbers side by side, and say which attributes you weighted on.

What Weighting Can’t Fix

Weighting can’t reach people who never use the product, so it doesn’t turn an own-user sample into a market sample. It also can’t fix a stratum with no respondents, and it can make a thin stratum count for a lot.

Cao, Koning and Nanda reached a related conclusion about early users: sampling bias, the gap between intended customers and the people who actually test, systematically hurt products aimed at underrepresented groups. A new category aimed at people outside your base is the same problem at a larger scale.

Duplicates and Comprehension

Two checks from the parent guide still apply. Read the comprehension item before any score, and report the comprehension rate per concept. And watch for repeat respondents, which retakes and link studies make possible, because a repeat counts twice in a cell. With your own users you know who answered, so you can usually remove duplicates cleanly.

Sprig Note: By default a user never sees the same study twice, which helps keep cells clean. Limitation: link studies are exempt from the waiting period, so duplicates are possible there, and test traffic in a development environment can see a study repeatedly.

Analyzing the Result

Analysis follows the parent guide: top-two-box, the full-scale mean and a two-proportion comparison between concepts, run against the minimum detectable difference. This guide adds one layer, which is comparing respondents with everyone eligible and, if you ran an outside arm, comparing the two audiences.

Comparing Own Users With the Outside Arm

Treat the audience gap as a result. If your own users score a concept well above the outside arm, the concept appeals to people who already chose you, and you’ve learned something you couldn’t learn from either arm alone. If both arms score it the same, the concept is less audience-dependent than you feared.

Run the two-proportion comparison on the audience gap the same way you run it between concepts, and check it against the minimum detectable difference for your actual cell sizes. A gap inside that difference is a tie, not a finding.

Sprig Note: Cross-tab and significance analysis run in the connected AI client, not in Sprig, and Theming is a Sprig capability for the reasons-why question. Limitation: Sprig documents no significance testing and no weighting, so the comparisons in this section happen on the export or in the client.

What Benchmarks Exist

No peer-reviewed study located in this guide’s research quantifies how much existing users' concept scores differ from those of people outside the customer base, so any correction factor you see quoted is unsourced. Don’t apply one, and don’t publish a score adjusted by a figure you can’t trace.

Build your own comparison instead. Run the same concept test on the same instrument with an outside arm at least once, and keep both results. That single paired study tells you more about your own audience gap than any published norm, and the parent guide explains how to build norms across studies.

Interpreting and Acting on the Result

Read the result in a fixed order, and state in the readout who it speaks for.

Read the Result in Order

Check comprehension first, then the gaps against the minimum detectable difference, then the distribution, then the reasons why. Add one step for this audience: check the response rate by stratum and note which strata are thin. A strong score driven by heavy users is a different result from a strong score across tenures.

What Each Outcome Licenses

A clear separation among concepts, with a representative spread of strata, licenses taking the winners to a prototype test. A strong own-user score with no outside arm licenses an extension decision, such as shipping a feature to people who already use the product. It doesn’t license a claim about new customers.

A weak score among your own users deserves weight, because these are the people most predisposed to like an extension. A low score from your most engaged users is hard to explain away. That’s a judgment, not a sourced result, but it’s a reasonable place to stop spending.

When the Two Arms Disagree

If your own users score a concept well above the outside arm, the concept appeals to people who already chose you. That supports an extension decision and nothing broader, and the reasons-why responses will usually show whether the appeal depends on familiarity with the current product.

If the outside arm scores higher, don’t dismiss it as noise until you’ve checked the gap against the minimum detectable difference. A real gap in that direction suggests your users are anchored on the current way of working, which is the pattern the steering objection in the critique section describes. Read the reasons-why responses from both arms before deciding.

If the arms agree, the concept is less audience-dependent than you feared, and the decision rests on the usual filter logic from the parent guide.

Say Who It Speaks for

Close the readout with one sentence naming the audience: "These results reflect active users on paid plans who triggered the export event, weighted by tenure, and don’t speak for people outside our customer base." A reader who sees that sentence knows what the number can carry, and the sentence costs nothing to write.

Readout Checklist

Use these lines as the structure of the readout, in order.

State the eligible population and its head count.
Report the response rate by stratum.
Name the attributes you weighted on, and show the weighted and unweighted scores together.
Say whether an outside arm was fielded, and report the arms separately.
Write the audience sentence: who the result speaks for.
Name what the result can’t support, such as demand among prospects or a new category.

Worked Example: Two Concepts, Two Audiences

This example is hypothetical, and its figures only show how the decisions connect. A project management product wants to test two ideas. The first is bulk editing for tasks, and the second is client invoicing.

Classifying and Choosing

Bulk editing changes something current users do today, and no outsider would see it as relevant, so it’s an extension. The audience is active users on paid plans, reached in the product with a trigger that fires after a user edits several tasks. The strata, written down before launch, are heavy, occasional and new users.

Client invoicing serves a need users don’t bring to the product today, and freelancers outside the customer base would want it. It’s a new category. The team fields it to its own users and to an outside arm, with identical instruments in the same window.

Reading the Results

Suppose bulk editing scores a 47 percent top-two-box unweighted and 41 percent after weighting by stratum, because heavy users answered most often. The team reports 41, shows both, and sends the concept to a prototype test with the audience sentence attached.

Suppose invoicing scores 24 percent among own users and 36 percent in the outside arm, with 200 respondents per arm. The parent guide’s tables put the minimum detectable difference near 14 points at that size, so the 12-point gap falls short of what that design detects reliably.

A two-proportion test on these numbers gives about p = 0.01, so treat it as a strong lead rather than a precise estimate. The sensible next step isn’t more own-user testing. It’s discovery research with the people the concept is for.

Running the Analysis with Claude or ChatGPT

The parent guide already carries the prompts for this analysis, so this guide doesn’t rebuild them. They load the export and identify which concept each respondent saw, clean and exclude, score the comprehension gate, compute top-two-box and means, test the differences with a second method as a check, theme the reasons why, and challenge your own interpretation. Start with the parent guide’s analysis chapter.

What to Add for Your Own Users

Add one column to the export before you run anything: the stratum each respondent belongs to, such as heavy, occasional or new. Ask the model to compute the response rate per stratum and the weights with code, not estimation, and to report the weighted and unweighted scores together. Give it the eligible population counts you saved on launch day, because the survey export can’t supply them.

For cutting results by segment, the simple cross-tab guide is the companion piece, and theming at depth belongs to the advanced cross-tab guide. The broader case for checking AI output is in how to analyze survey data with AI without sacrificing rigor.

What to Verify Before Reporting

Run any number you’ll report twice, and recompute the weights by hand for one stratum. Models can be agreeable, so never ask one to confirm a conclusion you already hold. And remember that your users' answers can contain personal information nobody asked for, so check what your assistant plan does with uploaded data before you share a raw export.

The Critique You Should Know

Three objections apply to testing concepts on your own users, and two bodies of evidence push back. Take them in order of how much they should change what you do.

The Sampling Bias Objection

Cao, Koning and Nanda studied an online platform where entrepreneurs launch new digital products and early users test them. In the abstract’s words, sampling bias is the gap between a startup’s target customers and the people who beta test its product.

Products aimed at women that launched on a typical day, when about 90 percent of platform users were men, grew 45 percent less a year later than more male-focused products (Management Science, 2024, vol. 70, pp. 7283 to 7307).

The scope matters. The setting was beta testing on a launch platform, not concept surveys, and the finding concerns groups underrepresented among early users. It still shows that who tests shapes what a team learns, and the effect can persist for a year.

The in-product version appears in the LinkedIn paper. Its authors couldn’t sample non-adopters, found response rates that varied several-fold between actively job-seeking and not-actively job-seeking members, and concluded they didn’t know how much bias remained after weighting. Both are different settings from yours, so use them as mechanisms, not as magnitudes.

The Existing-Customer Steering Objection

Christensen and Bower (Strategic Management Journal, 1996) argued that established firms' investment follows their powerful existing customers. Their line is the one quoted earlier: projects for technologies with no customers yet "languish for lack of impetus and resources."

Read it for what it covers. The evidence comes from the disk drive industry and concerns resource allocation, not survey sampling. But it supports a practical caution: if your concept test audience is only your current customers, the concepts that win are the ones current customers want, and a truly new category may lose by default.

The Asking Changes the Answer Objection

Chandon, Morwitz and Reinartz (Journal of Marketing, 2005) found that the intent-behavior correlation is "58% greater among surveyed consumers than it is among similar nonsurveyed consumers." Asking appears to change the behavior being predicted.

The study didn’t test in-product surveys, so the claim to take from it’s narrow. Users you survey often may be a different population from users you don’t, and a concept test on people you’ve asked before may overstate how well intent maps to action.

What Survives

Two bodies of evidence push back. Urban and von Hippel (Management Science, 1988) tested lead-user analysis in computer-aided printed circuit board design and found that concepts built from lead-user data were "strongly preferred by a representative sample of PC-CAD users." That shows some existing users carry information a general sample lacks.

The scope is industrial software and lead users, not all customers, so it supports choosing which of your users to ask, not asking all of them.

Kato, Kamei, Ootsubo and Ichiki (Journal of Business Analytics, 2023) ran a randomized controlled trial in the Japanese personal computer market and found that presenting the product and corporate brand gave the most accurate sales estimates. It supports testing with people who know the brand, though it never compared customers with non-customers.

The practitioner position is the one quoted at the start: recruit your actual customers, because they understand your product. That’s a fair argument for extensions. So the position this guide lands on is narrow: your own users are the right audience for concepts that extend what they already use, a weak audience for anything new, and a different population from your market whatever the concept.

Common Mistakes

These are the errors that turn an own-user concept test into a claim it can’t support.

Treating an own-user score as a market score. The result speaks for people who use your product and answered. It says nothing measured about anyone else.
Reporting a score without saying who answered. Without the audience sentence, a reader will assume the result covers more people than it does.
Choosing strata after seeing the data. A cut picked afterward is a search for a story, and it inflates false findings.
Pooling the own-user arm with the outside arm. The gap between arms was the reason for fielding both, and pooling erases it.
Triggering on one event and calling the result all users. An event trigger reaches people who did the event, so users who stopped doing it never see the concept.
Reading rotation as randomization. Documentation describes rotation when surveys share a trigger, and says response rates can differ.
Launching a test without checking the recontact window. Another team’s study can hold your users back, and the window is enforced only in production, so a development test hides the problem.
Putting your roadmap in the concept. Your users know your company, and an implied promise changes what they rate.
Reporting only the weighted number. Weighting corrects for chosen attributes only, so show both numbers and say what you weighted on.

Synthetic Respondents

Readers ask whether a synthetic audience can replace the outside arm. The parent guide covers the two relevant studies in detail, a preprint on semantic similarity ratings and a peer-reviewed paper on the perils of replacing survey data.

Both point the same way for this question. A synthetic audience built from your own research inherits that research’s sampling limits, so it can’t tell you about people your research never reached. Synthetic respondents may help screen a long list of concepts before you field anything, and they aren’t a substitute for an outside audience when the decision reaches beyond your customers.

How the Platform Supports Concept Testing on Your Own Users

Sprig supports this method in three places: reaching your users in the product, reaching people outside it, and analyzing what comes back. Depending on the plan and configuration, in-product surveys run on web apps and websites and on mobile apps, and email delivery reaches users who aren’t active.

The Design Agent is built to draft questions and flag unclear ones, and a concept test’s comprehension and intent items are the kind of instrument it can start.

The Synthesize Agent themes the reasons-why responses.

Theming is a Sprig agent capability, while the cross-tab and significance work happens in a connected AI client. The solution page for concept and prototype testing shows the broader workflow.

For the outside arm, Panels reaches people beyond your customer base. On G2, the platform holds 4.3 out of 5 across 199 reviews (retrieved October 15, 2026).

What Happens Outside the Platform

Some of this method happens outside the survey tool, and the readout should say so. Weighting and nonresponse adjustment aren’t documented in the product. Neither is significance testing between groups, so the comparisons run on the export or in your AI client. The eligible-population counts and strata come from your own product data.

And comparing results across studies, which is how you’d build your own audience-gap benchmark, means keeping each study’s export yourself.

Two constraints from the documentation affect planning. In-product studies reach only people with the SDK installed who trigger your event, and random assignment of respondents across concepts isn’t documented, so confirm the mechanism before relying on it.

Alternatives and Adjacent Methods

Pick the neighboring method by the question you can’t answer with own-user concept scores.

A panel concept test answers how people outside your base react. The parent guide shows the design.
Discovery interviews with non-customers answer whether a new category has demand. The AI-moderated interview guide linked earlier is the route.
A fake door or waitlist test answers whether anyone acts, because a real action is attached.
Gabor-Granger answers price, as covered earlier.
Conjoint and MaxDiff answer which attributes drive preference, with the conjoint and MaxDiff guides to hand.
Message testing applies when the variable is how you describe the idea, and the message testing guide covers it.
A voice of customer survey or a customer needs assessment comes earlier, when you don’t yet know what concepts to write.

A prototype test is the next study after a concept survives, and it’s the honest step after this method rather than an alternative to it.

Frequently Asked Questions

Can I Run a Concept Test on My Own Users?

Yes, when the decision concerns people who already use your product, such as an extension of a feature they rely on. Field it inside the product so you can target by behavior, and state who answered when you report. Don’t use own-user results to judge a new category or size demand.

Are My Own Users Biased?

Respondents can differ systematically from the users who don’t answer, and your users differ from your market. LinkedIn’s survey team documented response rates varying several-fold between actively job-seeking and not-actively job-seeking members. No peer-reviewed study located in this guide’s research quantifies the concept-score gap between existing users and outsiders.

When Do I Need an Outside Audience?

Use one when the concept is a new category, when the decision reaches people who don’t use your product, or when the decision is costly to reverse. Field the same instrument to both audiences in the same window, and report the arms separately.

Should I Field in the Product, by Email or by Link?

Field in the product when behavior should define the audience, by email when you need users who aren’t active, and by link when you want reach beyond your list. Link surveys are anonymous, so you can’t tie answers to what a user did.

How Many Users Do I Need?

Derive the number from the gap you need to detect, using the parent guide’s tables. Audience doesn’t change that arithmetic. It adds a second requirement: every stratum you’ll report needs enough respondents to support its own comparison.

Can I Weight the Responses?

Yes, by stratum, with the weight equal to a stratum’s share of eligible users divided by its share of respondents. The survey tool doesn’t document weighting, so do it on the export or in your AI client, and report weighted and unweighted results together.

Does the Recontact Window Affect a Concept Test?

Yes. The recontact waiting period is account-wide, so another team’s study can hold your users back, and it applies even to people who saw a survey without answering. Link studies are exempt, and the window is enforced only in production.

Can I Test a Prototype on My Own Users?

Yes, through a shareable link, and the documentation lists linking to one from an in-product survey. Recorded Tasks aren’t supported on mobile, and Prototype Tests are being deprecated as their functionality moves into Surveys, so confirm availability with support.

Should I Tell Users What the Survey Is About?

Say it’s a short study about ideas for the product, and stop there. Describing a concept’s benefits in the invitation primes every answer that follows, so let the concept statement do that work, and keep the invitation identical across concepts.

Can One Concept Test Set a Price?

No. A price inside a concept changes what respondents are rating. Use Gabor-Granger for price and keep the two studies separate.

What Can I Claim From an Own-User Result?

You can claim a reaction among the active users who answered, adjusted for the strata you weighted on. You can’t claim demand among prospects, a market forecast or a result for a new category. Write the audience sentence into the readout.