Introduction
Claude can run a full choice-based conjoint analysis: fit the model correctly, convert results into dollar willingness-to-pay by feature and segment, and verify its own numbers before reporting them. A plain prompt and an engineered prompt produce very different answers from identical data — the plain version loses a third of the dataset to a mislabeling trap and ranks price, the biggest real driver of choice, as the least important factor. This guide runs both versions side by side and shows exactly where each one succeeds or fails.
Code execution needs to be turned on for the verification step described below to run.
What is conjoint analysis?
Conjoint analysis shows people whole product configurations instead of one attribute at a time, then works backward from what they pick to figure out what each piece is actually worth. Ask someone to rate data privacy on a five-point scale and you learn that they nodded along. Show them a cheaper option with weaker privacy next to a pricier one with stronger privacy, repeat that trade-off across a dozen combinations, and you learn what privacy is worth to them in dollars.
This guide covers choice-based conjoint (CBC), the variant most survey platforms default to. Two other flavors exist: adaptive conjoint, which changes later questions based on earlier answers, and menu-based conjoint, where respondents build their own bundle instead of picking between fixed ones. CBC is the one you're most likely to run into.
When should you use conjoint analysis?
Use conjoint analysis when you need a defensible dollar figure attached to a specific feature, not just a popularity ranking — or when you're pricing a product with several moving parts instead of one fixed price point. It answers "how much would people give up to get this," which a simple preference survey can't.
A note on what's real here: the "Ideal AI Tool" in this guide doesn't exist. It's a generic composite built for this post, not a real product or roadmap. The 600 survey responses are synthetic too, generated to behave like a real conjoint study would. Nothing here is a real finding, customer, or usage number — it's a worked example, and only that.
What study is this guide based on?
This guide uses a choice-based conjoint study built around an AI tool for work, testing seven attributes: model capability, context window, integrations, data privacy, support level, response speed, and monthly price from $0 to $100. Six hundred respondents each worked through twelve choice tasks, three configurations per task plus the option to pick none of them.
The sample was split by who actually controls the purchase decision at their company: self-directed, manager-approved, procurement-controlled, or not sure.
Claude ran this analysis two very different ways, and that's the actual subject of the rest of this guide. Ask plainly, attach the data, and you get a real answer back in seconds: a model, some numbers, a confident-sounding summary. Engineer the prompt instead, and you get something slower to produce but checkable at every step. Both count as "Claude ran a conjoint analysis." Only one is something we'd let a pricing decision rest on.
If your own conjoint study already lives in Sprig, getting it in front of Claude doesn't take an export at all — Sprig MCP connects survey data, responses, and themes directly, so you're working from current results instead of a CSV pulled an hour ago.
What does a plain prompt get you?
A plain prompt — data attached, no engineering — silently dropped a third of the dataset and ranked price, the single biggest driver of choice, dead last. Two independent attempts produced the same two failures, plus no uncertainty on any reported number.
Here's what "plain" looked like:
I ran a conjoint analysis for an AI work tool. The data's attached. Can you run the analysis and tell me what matters most to buyers, what they'd pay for different features, and whether that changes based on who controls the purchase decision at their company?
What came back looked entirely reasonable on a skim: a model, a ranked list of what matters, a table of dollar figures. That's exactly the problem — nothing about it signals that anything went wrong.
The missing-data bug
A standard missing-values check turned up 7,209 nulls in the integrations column, out of 21,600 rows — 33% of the dataset. The reasonable-looking response was to drop those rows as incomplete. They weren't incomplete: they were the "no integrations" condition, with the category label "None" silently read as missing by the default CSV reader. Nothing in the output flagged that a third of the data had disappeared.
The price-ranking bug
Ranked by raw coefficient size with no scaling, price came out as the least important attribute in the entire study — 10th out of 10. A one-dollar coefficient sat next to a one-category jump and looked tiny by comparison, because nothing scaled either by the actual range it was estimated across. The single biggest real driver of choice in this data read as the smallest.
Underneath both bugs, the model never treated the three options within a task as competing against each other, and the categorical reference levels landed wherever alphabetical order put them — backwards for two attributes specifically.
Is this reproducible?
Yes. A second attempt, using a different library and different choices at every decision point, reproduced the same backwards reference levels and the same price-ranked-last result. Two unrelated implementations landing on the same structural mistakes independently is a stronger signal than either run alone — this isn't a fluke of one coding habit, it's close to a default.
One concrete number: procurement-controlled buyers came out valuing enterprise privacy somewhere between $85 and $99 a month more than standard, depending on which of the two runs you check — with no error bar on either figure. Directionally right. Not something to forward to a pricing meeting as a single number.
What does the engineered prompt actually say?
The engineered prompt fits a proper discrete-choice model instead of a naive regression, and requires Claude to independently re-verify every number before reporting it. It returns a ranked importance table, a dollar willingness-to-pay figure for every feature with uncertainty attached, and a segment-by-segment breakdown.
The 10-step process, in outline form:
- Load data defensively. Don't let "None," "NA," "N/A," or "NULL" be read as missing values. Check that every task has exactly one chosen alternative (or none_chosen = 1), never both or neither.
- Fit a discrete-choice model (conditional/multinomial logit). Dummy-code every attribute against explicit reference levels — don't rely on a tool's alphabetical default. Model each task's 3 concepts plus "none" against each other, not as 4 independent yes/no rows.
- Calculate relative importance as each attribute's utility range divided by the sum of all ranges, in code, at full precision.
- Check segment sample sizes before splitting by purchase authority. Flag any segment under 84 respondents (the floor for this design: 500 × 6 price levels ÷ (12 tasks × 3 concepts) = 83.3) as too small to model separately; keep those respondents in the pooled model only.
- Confirm each price coefficient is negative and non-zero before converting anything to dollars. Convert utility differences to dollars only for models that pass this check, with standard errors on every figure.
- Compare model-implied WTP to a stated-preference check using a formal significance test (gap ÷ combined standard error ≥ 1.96). A large-looking gap that doesn't clear that bar gets reported as directional, not a finding.
- Recompute every number independently, through a second, genuinely different implementation, before reporting anything final.
- Present an importance table and a WTP table, with uncertainty on every figure and any under-sized segment flagged explicitly.
- Build a grouped bar chart of dollar WTP by attribute and segment, if clearer than the table alone.
- Summarize in 2–3 sentences, explicit about which segment differences the sample size and standard errors actually support.
Full prompt text, adaptable to your own attributes and filename, is available on request — every column name has to match your file's literal header row, not a spreadsheet column letter.
What did the engineered prompt find?
Price accounted for 35.8% of the variation in choice — more than any other attribute — with model capability a distant second at 21.3%. Enterprise-grade privacy is worth $95 a month to procurement-controlled buyers and $7 to self-directed ones; dedicated account management is worth $67 to procurement and, statistically, close to nothing to self-directed buyers.
The "not sure" segment came in at 30 people, below the 84-respondent floor, so it was flagged and left in the pooled numbers only rather than modeled on its own.
The design also let the guide check its own privacy/support finding rather than just report it: Frontier-tier model capability — an attribute nobody built to vary by segment — shows a real but much smaller gap, roughly 2x between procurement and self-directed buyers, which is what plain price sensitivity alone would produce. Privacy and dedicated support are 13x and 25x apart between those same two segments, not 2x. Whatever's driving that gap, it's well past what price sensitivity could explain on its own.
A stated-preference question gave a second, independent check: people's guess at what they'd pay for the fully-loaded bundle ran higher than the model's estimate in every segment, consistent with the well-known tendency to overstate hypothetical willingness to pay. That gap holds up statistically for the overall sample and two of the three modeled segments. For procurement-controlled specifically, the gap looks similar in size but the segment is smaller and the estimate noisier — it doesn't clear the same significance bar, and is reported as directional rather than confirmed.
Which prompt should you trust for a pricing decision?
Trust the engineered prompt for any number you'd put in a deck; trust the plain prompt only for a fast directional read. Both got the core direction right — procurement values privacy and support more than self-directed buyers — but only the engineered version gives you a number with a margin of error attached.
| | Plain prompt | Engineered prompt |
|---|:---:|:---:|
| Missing-data handling | Silently dropped a third of the dataset (or invented a phantom category on a second attempt) | Reads defensively, catches the collision, keeps all 21,600 rows |
| The "none of these" option | Never modeled | Modeled explicitly, with its own term |
| Reference levels | Backwards for privacy and speed; none of the 6 attributes used the intended reference | Explicit, stated, and checked before reporting |
| "What matters most" | Price ranked dead last, both attempts | Price ranked 1st of 7 |
| Uncertainty | None on the first attempt; present but attached to a misspecified model on the second | Every dollar figure carries a confidence interval |
| Thin segment (n=30) | Modeled and reported like any other | Flagged, excluded from its own estimate, kept in the pooled number only |
| Verification | One pass, nothing rechecked | Two independent implementations, required to match |
| Reproducibility | Two honest attempts, two different dollar figures for the same finding | Same data, same result, every time |
Best for a plain prompt: a fast, first-look sanity check on direction — "does procurement value privacy more than self-directed buyers?" — when you don't need a specific number and can tolerate checking it manually.
Not ideal for a plain prompt: any number that goes into a pricing decision, a board deck, or anything a stakeholder will hold you to later. Without engineering the prompt, you can't tell which of two possible dollar figures for the same finding you've actually landed on.
What should you check in your own prompt?
Four checks catch most of what goes wrong in an unengineered conjoint analysis:
- Assume alphabetical reference levels until proven otherwise. The same bug showed up in both pandas' get_dummies and statsmodels' formula interface — both default to alphabetical order with no warning.
- Scale before ranking "what matters most." Raw coefficient size silently punishes any variable measured on a different scale than the others; a dollar and a category jump aren't the same unit.
- Name your "None" categories explicitly in the prompt. Default CSV readers treat literal level names like "None," "NA," or "N/A" as missing data, and nothing will warn you it happened.
- Run the plain version once, honestly, before assuming engineering was overkill. The gap between the two is usually bigger than it looks from the outside.
FAQ
Why not just ask Claude to run a conjoint analysis directly? A plain prompt lost a third of the dataset to a mislabeled category and ranked price — the biggest real driver of choice — as the least important factor. Asking Claude for a conjoint analysis gets you something back in seconds either way; whether it's something you can act on depends on what you asked for.
Is this a Claude-specific problem, or would other AI tools hit the same issues? The two failures that replicated across two independently-written attempts — "None" read as missing data, and reference levels dropped alphabetically — are properties of the Python libraries most AI coding assistants default to (pandas get_dummies, statsmodels' formula interface), not something specific to Claude's training. Any tool writing similar code in similar libraries would plausibly hit the same landmines.
Can I use this prompt on my own real conjoint data? Yes. Swap in your own attributes, levels, and filename, and recalculate the sample-size floor for your own design instead of reusing 84 — that number comes from a 6-level price attribute across 12 tasks with 3 concepts each.
What if my study doesn't have a segment variable or a stated-preference calibration question? Drop the segment-sizing step if you're not segmenting, and drop the stated-preference comparison if you didn't ask a calibration question. Nothing else in the process depends on either being present.
How do you know the "plain" version wasn't set up to fail? It was run twice, independently, changing the approach at nearly every decision point — dropped incomplete-looking rows once and filled them in the second time, used scikit-learn once and statsmodels the second time. Both attempts failed the same two ways regardless.
Does this mean a quick, unengineered Claude analysis is never worth trusting? No. Asked directly whether the finding changes by segment, the plain version got the direction right — procurement valuing privacy and support more than self-directed buyers — without being told to look for a segment effect. For a fast directional read, that's genuinely useful. The gap shows up once you need a specific, defensible number rather than just a direction.
