How to Run: Advertising Effectiveness Survey

Introduction

An advertising effectiveness survey, often called a brand lift study, measures what a campaign did to awareness, ad recall or consideration by comparing two groups: people who saw the ads and a randomly held-out control group who were eligible but didn’t. Randomize the holdout before launch, size each group from the smallest lift you need to detect, and field one identical questionnaire to both groups in the same window.

Last updated October 2026.

What an Advertising Effectiveness Survey Is

An advertising effectiveness survey is a controlled comparison of brand metrics between an exposed group and an unexposed control group, run while a campaign is live or shortly after it ends. It answers one question: did this campaign move what people remember or think about the brand, relative to what they would have thought anyway?

The method sits between two neighbors. A message testing survey screens creative before launch under forced exposure and ranks the options. A brand tracking survey measures the same metrics on a fixed cadence to detect drift. A lift study attributes a movement to one campaign. That attribution holds only when the control group differs from the exposed group in exposure and nothing else.

Pre-Testing Versus Post-Testing

Advertising research splits into two jobs that get confused. Pre-testing evaluates creative before launch and asks which ad is better. Post-testing evaluates a live campaign under natural exposure and asks whether the ad that ran did anything. A pre-test needs forced exposure and a ranked winner, while a post-test needs a control group and a measured difference.

Absolute, Relative and Headroom Lift

Three definitions dominate reporting, and Google’s published Brand Lift definitions are the clearest reference. Absolute lift subtracts the baseline group’s positive rate from the exposed group’s. Relative lift divides that difference by the baseline rate. Headroom lift divides it by one minus the baseline rate.

Report absolute lift as the headline. Relative lift inflates small bases, so a move from 4 to 6 percent reads as a 50 percent relative lift.

When to Use an Advertising Effectiveness Survey

Use a lift survey when a campaign is meant to change a memory or an attitude and someone needs to know whether it did before the next budget decision. Brand launches, category-entry campaigns and lifecycle campaigns fit, because their goal is awareness, consideration or intent rather than an immediate sale.

Four conditions make the method a strong choice:

The campaign reaches enough people to fill the cell sizes your target lift requires in each group
A holdout can be withheld from every channel carrying the campaign
The goal is awareness, recall, consideration or intent, not an immediate purchase
The decision turns on direction and approximate size, not on revenue

Owned channels suit the method well. A subscription app sending a reactivation email to lapsed users can withhold the email from a random slice of the list, say 20 percent, and survey both groups a week later, which needs no ad platform at all. Email and in-app messages are the easiest places to build the counterfactual yourself.

Campaign Types That Fit

Launches and category-entry campaigns aim at awareness, so their primary metric is usually aided awareness. Lifecycle campaigns aim at existing customers, so consideration or intent often carries the decision. Sponsorships and broad-reach flights fit too, but they usually need the matched-market design covered later, because nobody can randomize a billboard by person.

Much of the published lift guidance assumes the advertiser buys media on someone else’s platform. If you control the list, you don’t have to wait for a platform study.

When Not to Use an Advertising Effectiveness Survey

A lift survey answers an attitudinal question about one campaign. It loses to other instruments on seven dimensions, and each limitation below names the instrument that does the job better. The first two are the ones readers most often need.

It Can't Tell You What the Campaign Was Worth

A lift survey measures an attitude or a memory, not a purchase. The better instrument is a conversion lift test or a geo experiment with sales as the outcome, or a marketing mix model that connects spend to revenue.

Lewis and Rao (2015), analyzing 25 large field experiments in the Quarterly Journal of Economics, found that "the median confidence interval on return on investment is over 100 percentage points wide." Measuring sales effect is hard even for the teams best equipped to do it, and a survey can’t shortcut that problem.

It Can't Tell You Which Creative or Claim Did the Work

An in-market lift study compares exposure with no exposure. Unless the campaign randomized creative versions across people, the study can’t attribute the effect to one execution. The better instrument is pre-launch monadic message testing, which isolates one message per cell, or MaxDiff across claims when you need a stable ranking of many items.

It Can't Tell You Whether the Effect Lasts

A single post-campaign read is a snapshot. To judge durability, date the campaign against a brand tracker and read the tracker’s metrics inside a market-response model. The brand tracking guide covers that design, including the evidence that consideration earns its place as a leading indicator of sales.

It Says Nothing About People Who Don't Answer Surveys

Lift is measured on responders. Lavrakas (2010), in a review commissioned by the Interactive Advertising Bureau (IAB), found that "the response rates that almost all IAE studies garner are less than 5% and in many cases are far less than 1%." The better instrument is a behavioral outcome, such as site visits, sign-ups or purchases, measured on the full exposed and control populations from the same randomized split.

It Can't Explain Why People Were Unmoved

Awareness and consideration batteries measure incidence, not reasoning. The better instrument is a moderated or AI-moderated interview, which answers why from a small number of participants. Interviews are qualitative instruments in a different category from survey platforms. They answer a different question rather than a better version of the same one.

It Can't Rank Channels Unless Channels Were Randomized

A single exposed-versus-control comparison measures the whole campaign. When the campaign ran on television, social and email at once, nobody can divide the lift among them afterward. The better instrument is a multi-cell holdout that withholds each channel from a separate random group, or a marketing mix model when the channels can’t be withheld independently.

Self-Reported Exposure Can't Confirm That Exposure Happened

Recall is an imperfect proxy for exposure. The better instrument is log-level exposure data from the delivery system, which records who was served the ad. The counter-evidence is real, and the critique chapter covers it: self-reported recall does track actual digital exposure in published validation work.

Study Design and the Control Group Fork

The design of a lift study is a choice about where the counterfactual comes from. Five options exist, and the evidence ranks them clearly. Work down the list and take the first branch your media setup allows.

Branch 1: A Randomized Holdout at the Ad-Delivery Layer

A randomized delivery holdout is the strongest design available. The ad platform withholds the campaign from a random slice of eligible people and surveys both slices, so the only systematic difference between the groups is exposure. Google’s Brand Lift compares "People who have seen your ads" with "People who were eligible to see your ads, but didn’t see them because the ad was not shown."

Johnson, Lewis and Nubbemeyer (2017), writing in the Journal of Marketing Research, formalized the ghost ads method that identifies those control counterparts. They showed that "advertisers can measure ad lift just as precisely while spending at least an order of magnitude less" than public service announcement and intent-to-treat designs.

The platform runs this branch end to end, inside its own inventory, and no general survey platform sits on it. A survey fielded alongside it can’t see the platform’s holdout, so it can answer diagnostic questions but can’t measure lift. If the platform offers a randomized study, use it for the lift figure.

Branch 2: A Randomized Holdout on a Channel You Control

An owned-channel holdout is a strong design that platform-focused guidance rarely covers. Email, in-app messages, push notifications, direct mail and uploaded customer-match audiences all let the advertiser decide who receives the campaign. Randomize the holdout before launch, suppress it on every channel carrying the campaign, and field one survey to both groups in the same window.

Assignment was random and recorded, so the comparison is valid, but it compares assigned groups rather than exposed people. Unopened sends dilute the lift. Never restrict the exposed group to people who opened, because that reintroduces the self-selection of Branch 4. The cost of this design is discipline: the holdout has to stay suppressed for the full campaign, retargeting included.

Branch 3: A Matched-Market Holdout Measured Before and After

A matched-market design runs the campaign in some regions, holds out comparable regions, and surveys both before and after the flight. The analysis reads the difference in differences: the change in exposed regions minus the change in control regions. It fits channels that can’t be randomized person by person, such as television, out-of-home, audio and broad-reach digital buys.

Two costs belong upfront. Comparing four cells roughly doubles the respondents each cell needs relative to a two-cell post-only comparison. And respondents in the same region share everything the region shares, so with a handful of regions the effective sample is closer to the number of regions than the number of respondents.

Branch 4: Self-Reported or Matched Exposure (Not Recommended)

Comparing people who say they remember the ad with people who say they don’t is a common default in agency work, and it’s the weakest design. People who already use a brand notice its advertising more, so the "aware" group differs before any ad runs.

Post-hoc matching doesn’t rescue it. Gordon, Moakler and Zettelmeyer (2023) analyzed 663 Facebook experiments with over 5,000 user-level features. Median true lifts were 29, 18 and 5 percent for upper, middle and lower funnel outcomes, against non-experimental estimates of 83, 58 and 24 percent using double machine learning, and 173, 176 and 64 percent using stratified propensity score matching.

The authors concluded that "we are unable to reliably estimate an ad campaign’s causal effect." If matching fails with platform-grade data, it fails with a demographic screener.

Branch 5: Before and After With No Control

A pre-post comparison with no control group is a brand tracker, not an effectiveness study. It detects that a metric moved and can’t separate the campaign from seasonality, competitor activity or a category shock.

The Recommended Default

The committed recommendation, one sentence per situation:

Randomize a holdout before launch whenever you control delivery, and survey both groups together
Use a platform’s randomized study for the lift figure whenever you buy media that offers one
Run a matched-market pre-post only when neither randomized option is available
Refuse to report self-reported exposure comparisons as lift, whatever the deadline

The Control Group Decision Tree

Answer each question in order and stop at the first yes.

The fourth row is the honest exit. A campaign with no counterfactual is unmeasured, and saying so protects the credibility of the next study.

Where Platform Constraints Change the Design

Platform constraints belong in the design chapter because they remove branches. Sprig can’t observe ad exposure. It documents no impression pixel, no ad-server connection and no ad-platform integration, so Branch 1 is closed to any survey Sprig fields.

The reader brings the assignment to Branches 2 and 3, and the arm has to travel in the data: a link parameter or a user attribute, since group membership doesn’t appear as a column in the response export.

Sprig, and the constraint: Sprig supports Branch 2 directly. Teams can upload the exposed list and the holdout list as CSV groups, carry the arm as an additional attribute column, or stamp it on a survey link as a custom parameter, then target one survey at both.

The constraint is that Sprig trusts the list it’s given. If the holdout wasn’t randomized before launch, the study inherits that bias, and the documentation states no group size limits.

Branch 3 depends on how finely a panel can target geography, which the audience chapter takes up.

Writing the Lift Instrument

A lift instrument is short, identical for both groups, and ordered so that nothing shown early contaminates a measure asked later. Unprompted questions come before prompted ones, and brand-level questions come before ad-level ones. Build the instrument around the campaign’s single objective. A launch campaign’s primary metric is usually aided awareness, and a consideration campaign’s is consideration.

The Frozen Lift Questionnaire

Lift this without modification, and keep the order, because the order is part of the specification:

Screener on category purchase or role, which never names the brand
Unaided brand recall in the category, as open text
Aided brand awareness against a randomized list of the brand and three to six competitors (a design choice, not a published rule)
Aided ad recall, showing the campaign creative
Aided ad recall for a plausible creative that never ran, as a false-recall check
Familiarity with the brand, on a five-point scale
Favorability toward the brand, on a five-point scale
Consideration for the next purchase in the category
Purchase or sign-up intent, on a five-point scale
Channel where the ad was seen, asked only of those recalling it
Demographics, last

The sequence moves from least to most prompted. Each step exposes the respondent to more brand information, so reversing any two items leaks information into the later measure.

Why the False-Recall Item Earns Its Space

Item 5 is a design option, not a published standard, and it earns its place. A creative that never ran measures directly how many respondents claim to remember an ad they couldn’t have seen.

The control group’s answer to item 4 measures the same thing from another angle, since the control group never saw the real ad. Reporting both numbers shows how much of the exposed group’s recall is memory and how much is agreement.

Question Wording Rules for Lift Studies

Five wording rules prevent the defects that most often break a lift comparison:

Name the category before any brand appears on screen
Randomize the brand list order while keeping the list itself fixed
Show the creative at the size and format people saw it in market
Ask about the brand before asking about the advertising
Keep every label identical across groups, including punctuation

An edited label is a new question. Even a one-word change between the exposed and control versions invalidates the comparison for that item.

Showing the Creative for Aided Recall

Show the creative as people encountered it: a video as a short clip, a display ad at its served size, an email as a screenshot of the inbox view.

Remove the brand name from the creative where possible, because a visible logo turns a recall question into a brand-recognition question. Where the campaign ran several executions, show the one with the most delivered impressions, or show each to a random subset and report recall per execution.

Length Discipline

Hold the instrument to the fewest items that serve the primary metric. Break-off that differs between groups changes who completes, so every added item needs a decision it would change. The 11 items above are a design option for this guide, not a published standard length.

Sprig, and the constraint: Sprig’s Field Agent generates AI follow-up questions in real time, which suits the channel probe in item 10. The constraint is balance. Follow-ups add length unevenly when the control group has less to say, so turn them off for the lift study and keep both groups on an identical instrument.

Scale and Scoring Choices

Score each lift metric as a proportion rather than a mean. The positive response is the top box on awareness and recall, and the top two boxes on five-point familiarity, favorability and intent scales. Proportions make absolute, relative and headroom lift directly computable and make the sample-size arithmetic tractable. Means on ordinal scales often hide where the movement happened.

Top-Box Versus Top-Two-Box

Top-two-box is a common default for favorability and intent because it produces larger, more stable bases. Top-box is the more conservative choice for intent, because it counts only the strongest stated commitment. Goldfarb and Tucker (2011) scored a five-point purchase-intent scale on the single response "very likely to make a purchase." Pick one scoring rule per metric before fielding and don’t switch after seeing the data.

Scoring Aided Recall and Awareness

Score aided ad recall as the share answering yes to the real creative. Report the false-recall item beside it instead of netting the two, because netting hides the size of the false-recall share, which no published benchmark sizes. Score aided awareness as the share selecting the brand from the randomized list. Keep the list length fixed across groups, since a longer list lowers every brand’s selection rate.

Scoring Consideration

Score consideration as the share naming the brand among those they’d seriously consider for their next purchase in the category. Ask it as a multi-select from the same randomized brand list used for aided awareness, so the two measures share a denominator.

Scoring Unaided Recall

Unaided recall is open text and needs coding into brand mentions before it can be scored. Code both groups with the same codebook, built before either group’s responses are read. Fix the brand list first and have the model apply it.

Hill and colleagues found models match human coders when applying a fixed list and fall short when discovering one. You remain responsible for checking a sample of coded mentions against the verbatims.

Sample Size and Precision for a Lift Study

Size each cell from the smallest lift you need to detect, the baseline rate of the metric in the control group, and the confidence and power you’re willing to accept.

No professional body publishes a defensible per-cell rule for survey lift studies, so this chapter gives the derivation instead of a borrowed number. The same two-proportion arithmetic produces the per-cell figures in the message testing guide, and the 1,377 and 356 below match it.

The closest thing to a published rule is the IAB’s 2011 best-practices summary (retrieved October 15, 2026), which advises researchers to "optimally strive for 200 respondents," with "100 responses for smaller sites." Google publishes response guidance for its own product: "about 2,000 responses per lift metric" for high-performing campaigns and "4,100 responses per lift metric" at its recommended budget minimum.

Google doesn’t say whether its counts are per group or combined. Read as per group, they land where the arithmetic below puts a detectable lift of two to four points, and the IAB’s 200 doesn’t.

The Assumptions Behind the Tables

Every table here assumes two independent cells of equal size, a two-sided test at 95 percent confidence, 80 percent power, no weighting and no design effect. The tables are derived for this guide, and each cell was recomputed by simulation. The sample needed per cell for a difference between two proportions is:

n = (z₁ × √(2 × p̄ × (1 − p̄)) + z₂ × √(p₀ × (1 − p₀) + p₁ × (1 − p₁)))² ÷ (p₁ − p₀)²

Here p₀ is the control rate, p₁ is the exposed rate, p̄ is their average, z₁ is 1.96 for 95 percent confidence and z₂ is 0.84 for 80 percent power. Round each result up.

Detectable Lift by Cell Size

The first number in each cell is the observed lift that reaches significance at 95 percent. The second is the true lift the study detects 80 percent of the time.

| Respondents per cell | Baseline 10 percent | Baseline 30 percent | Baseline 50 percent | |:--------------------:|:-------------------:|:-------------------:|:-------------------:| | 100 | 8.3 / 15.0 points | 12.7 / 19.3 | 13.9 / 19.3 | | 200 | 5.9 / 10.0 | 9.0 / 13.5 | 9.8 / 13.8 | | 500 | 3.7 / 5.9 | 5.7 / 8.4 | 6.2 / 8.8 | | 1,000 | 2.6 / 4.1 | 4.0 / 5.9 | 4.4 / 6.2 | | 2,000 | 1.9 / 2.8 | 2.8 / 4.1 | 3.1 / 4.4 | | 4,100 | 1.3 / 1.9 | 2.0 / 2.9 | 2.2 / 3.1 |

At 200 respondents per cell and a 30 percent baseline, a lift study detects a true lift of about 13.5 points and nothing smaller. The two disclosed reference points, about 3 points on favorability from DISQO and about 0.7 points on intent from Goldfarb and Tucker, sit well below that. The table uses the standard error at the control rate, so treat each figure as a planning approximation.

Cell Size by Target Lift

The first figure uses 95 percent confidence. The figure in parentheses uses the 80 percent two-sided interval that Google’s Brand Lift reports.

| Control baseline | 2 points | 3 points | 5 points | 8 points | 10 points | |:----------------:|:-------------:|:-------------:|:-----------:|:---------:|:---------:| | 10 percent | 3,841 (2,206) | 1,774 (1,019) | 686 (394) | 295 (169) | 199 (115) | | 20 percent | 6,510 (3,739) | 2,943 (1,690) | 1,094 (628) | 447 (257) | 294 (169) | | 30 percent | 8,394 (4,821) | 3,763 (2,161) | 1,377 (791) | 550 (316) | 356 (205) | | 50 percent | 9,806 (5,632) | 4,356 (2,502) | 1,565 (899) | 609 (350) | 388 (223) |

Detecting a three-point lift at a 30 percent baseline takes about 3,800 respondents per cell. Lower baselines need less sample for the same absolute lift.

How Small Real Effects Can Be

The peer-reviewed reference point is sobering. Goldfarb and Tucker (2011) studied 2,892 randomized display campaigns and found that exposure to a single ad increased the share "very likely to make a purchase" by 3.6 percent relative to a mean of 0.20, which is roughly 0.72 percentage points absolute (3.6 percent of 0.20, our derivation from their figures).

Detecting a 0.72-point lift on a 20 percent base needs about 49,000 respondents per cell under the formula above. Their campaigns averaged 852 survey takers each, which is why the authors pooled thousands of campaigns instead of reading any single one.

The Multiple-Metric Problem

A standard lift battery tests five or six metrics, and each test carries its own false-positive risk. With six independent metrics and no true effect anywhere, the chance of at least one false significant result is 26.5 percent at 95 percent confidence. Correlated metrics lower that figure somewhat.

Google reports lift at 80 percent two-sided intervals, and at that convention the same chance rises to 73.8 percent. Name one primary metric before fielding and treat the rest as directional or corrected.

Matched-Market Designs Need About Twice the Sample

A difference-in-differences read compares four cells: exposed and control regions, before and after. The variance of that comparison is roughly twice the variance of a two-cell post-only comparison. At a 30 percent baseline and a five-point lift, that means about 2,750 respondents per cell instead of 1,377. Count the regions as well as the respondents, because a design with four regions has very few independent units.

No Universal Sample Floor

This guide publishes no minimum sample that applies to every study. The right number depends on the baseline and the smallest lift worth acting on, and the tables let you derive it for your own campaign.

Audience, Targeting and Screening

A lift study surveys the people the campaign targeted, split into exposed and control groups that were eligible for the same campaign. The control group has to meet the campaign’s targeting rules, or the comparison measures targeting instead of advertising. Lavrakas (2010) found that comparison groups in many industry studies "often aren’t part of the target population."

Screen Without Naming the Brand

Screen on category purchase, role or the campaign’s audience definition before any brand appears. A screener that mentions the advertised brand selects people who know it and inflates every awareness measure downstream. Business-to-business campaigns need a role screener as well as a category screener, since awareness among people who can’t buy is a different metric from awareness among buyers.

Match the Control to the Campaign's Targeting

On owned channels, both groups come from the same eligible list, so they match by construction. In matched-market designs, the control regions should resemble the exposed regions on the variables that drive the metric: category penetration, prior brand share and demographic mix. Match regions on the pre-campaign baseline of the primary metric, measured in the before wave, not on population alone.

Customer-Match Audiences on Paid Platforms

Uploaded customer lists on paid social and search platforms sit between owned and paid media. The advertiser controls who is on the list, so randomly splitting it before upload creates a valid holdout even when the platform offers no lift study. The constraint is suppression: the holdout half has to be excluded from every other campaign targeting the same people, including lookalike audiences built from the full list.

Sprig, and the constraint: Sprig Panels offers over 300 targeting filters, location among them, and multiple panels can run on one survey, which could support a matched-market pre-post if the location filters reach the granularity you need. Panels is available to Enterprise teams on link surveys only.

The constraint for this method is geography: the documentation lists "Location" as a category but doesn’t say whether targeting reaches state, metro or postal-code level, or which countries are covered. Panel respondents also can’t be matched to individual exposure, so geography is the only lift design Panels supports. Confirm both gaps before committing to Branch 3.

Fielding and Delivery

Field both groups from the same source, in the same window, with the same incentive and the same survey environment. Any difference in how the groups are reached becomes part of the measured lift.

The IAB’s 2011 guidance adds one environmental rule: surveys "should be given in an environment where measured brand ad is not visible." A survey that shows the campaign’s banner on the page where it loads has re-exposed the control group.

Owned-Channel Fielding

On owned channels, deliver the survey by the same route to both groups, usually an email or an in-app prompt. The invitation must not reference the campaign, or it becomes an exposure for the control group.

In-product surveys carry one caveat for brand questions: the product itself is exposure to the brand, so absolute awareness runs high in both groups. The difference between groups stays valid, but the levels shouldn’t be compared with panel-based studies.

The Holdout Integrity Checklist

Run these checks in launch order, because each one is cheapest to fix before the step that follows it:

Randomize the holdout from the full eligible list before any campaign asset is scheduled
Record each person’s group in a file that the survey and the delivery system both read
Suppress the holdout on every channel carrying the campaign, including retargeting
Exclude the holdout from lookalike audiences built from the eligible list
Freeze the instrument and confirm both groups receive the identical version
Schedule both groups' survey invitations for the same days, from the same sender
Confirm the survey page shows no campaign creative or brand advertising
Audit the delivery logs mid-flight for any holdout member who was served the campaign

Items 3, 4 and 8 are the easiest to skip. Each protects the one property that makes a lift study valid: the control group wasn’t exposed.

Keep the Groups From Mixing

Forwarded links, shared inboxes and retargeting pixels are the usual routes for contamination. Assign the group at the respondent level and check the assignment against the delivery log before analysis.

Sprig, and the constraint: Sprig survey links accept user_id and up to 25 custom parameters totaling 1,000 characters, and they surface in the export as customMetadata. An email or messaging tool can stamp the group, campaign and wave on every response.

The constraint is capacity and leakage: the 1,000-character cap is a hard limit, and a link forwarded from an exposed inbox to a control inbox carries the wrong group with it. Sprig recommends turning on Bot Detection when embedding surveys in email, and lift studies should follow that advice.

Timing and Cadence for Lift Surveys

Survey after the campaign has reached its planned frequency and before the flight ends, then optionally again a few weeks after it ends. Surveying on day one of a flight measures reach, not effect.

In-Flight Timing

Open the survey once the exposed group has had the chance to see the campaign several times, as defined by the media plan’s frequency target.

Keep it open long enough to fill both cells, and close it before the flight ends so the exposed group is still being reached. Field both groups over the same days of the week and for the same length of time, because early responders differ from late ones.

Post-Flight Timing

Johnson, Lewis and Nubbemeyer, in a 2017 working paper analyzing 432 display field experiments on behavioral outcomes, found that "most campaigns have a modest, positive carryover four weeks after the campaign ended." That result covers site visits and conversions, not survey metrics, so treat it as a reason to measure after the flight as well as during it.

It isn’t a decay rate. A second wave fielded to fresh respondents from both groups shows whether the lift persisted. Compare the post-flight lift with the in-flight lift, not with zero.

Quality Control and Data Hygiene

Run the same hygiene checks on both groups and record the results beside the lift. A difference in data quality between groups looks exactly like a difference in brand metrics. Five checks catch most defects:

Compare completion rates between groups before reading any metric
Compare the demographic profile of completes between groups against the targeting
Remove speeders and straight-liners with the same rule applied to both groups
Check the false-recall item for implausibly high agreement in either group
Confirm every response carries a valid group assignment from the delivery log

Removing Low-Quality Responses

Write one rule for speeders and straight-liners before fielding and apply it to both groups. The exact cutoff matters less than applying the same one everywhere. Record how many responses each rule removed in each group, since a removal rate that differs sharply is a warning that the groups were reached differently.

The completion-rate check comes first because differential break-off changes who is being compared. If the exposed group completes at a noticeably higher rate, the lift is partly a composition effect.

Weighting to the Delivered Audience

Weight the completes back to the campaign’s delivered audience when the profiles diverge. The IAB’s 2011 summary recommends weighting "to actual campaign delivery and demos." Report weighted and unweighted lift side by side so the adjustment stays visible.

Analysis: Calculating Lift and Testing It

The core calculation compares two proportions for each metric: the exposed rate against the control rate, tested with a two-proportion z test. The message testing guide walks through that test in full. Compute absolute lift first, test it, and only then derive relative and headroom lift from the same numbers.

Step 1: Compute Each Group's Rate

For each metric, divide the positive responses by the valid responses in that group. Exclude blanks and skipped items from the denominator instead of counting them as negative. Say 312 of 1,200 control respondents give the positive answer on consideration, so the control rate is 26.0 percent. If 366 of 1,180 exposed respondents do, the exposed rate is 31.0 percent.

Step 2: Compute Absolute Lift and Test It

Absolute lift is the exposed rate minus the control rate, here 5.0 percentage points. The test statistic uses the pooled proportion across both groups:

z = (p₁ − p₀) ÷ √(p̄ × (1 − p̄) × (1 ÷ n₁ + 1 ÷ n₀))

In the example, the pooled rate is 28.5 percent, the standard error is about 1.85 points, and z is about 2.71. That clears the 1.96 threshold for 95 percent confidence.

Step 3: Derive Relative and Headroom Lift

Relative lift divides the absolute lift by the control rate: 5.0 ÷ 26.0 is about 19 percent. Headroom lift divides it by the control share not yet positive: 5.0 ÷ 74.0 is about 6.8 percent. Report all three with absolute lift first.

Step 4: Report the Interval, Not Only the Verdict

Report a confidence interval for the absolute lift beside every significance verdict. In the example, the 95 percent interval runs from roughly 1.4 to 8.6 points using the unpooled standard error. An interval that wide tells a stakeholder more than a verdict does: the campaign moved consideration, and the plausible size of that movement spans a factor of six.

Step 5: Difference in Differences for Matched Markets

For a matched-market design, compute the change in each region group and subtract: exposed after minus exposed before, minus control after minus control before. Test that difference with a standard error that combines all four cells. With few regions, add a region-level read that treats each region’s change as one observation. If the two disagree, report the region-level result as the honest one.

Benchmarks and What a Good Lift Looks Like

We found no independent, methodologically disclosed cross-industry benchmark for brand lift. The largest commercial dataset we found that discloses its scale, DISQO’s medians from approximately 1,650 campaigns fielded from March 2021 through December 2024, doesn’t say how its control groups were formed.

The absence is the answer, and it’s more useful than a borrowed norm. A cross-industry average mixes channels, creative quality, brand sizes and control-group methods, so it isn’t a target for any particular campaign.

What the Disclosed Sources Say

DISQO reports that "brand favorability moves about 3 points after a typical ad campaign," and describes the comparison as exposed against unexposed consumers (DISQO, retrieved October 15, 2026). The page doesn’t say whether assignment was randomized, and the figure sits in a chart image. That’s a vendor-published median, not a controlled estimate. The peer-reviewed reference point is smaller.

Goldfarb and Tucker (2011) found roughly a 0.7-point absolute increase in top-box purchase intent from a single display exposure, across 2,892 randomized campaigns. That result dates to 2011-era display, so read it as evidence of how small real effects can be, not as a norm.

Every other "average brand lift" figure we found, across agency blogs, glossary pages and vendor case-study roundups, cites no dataset. This guide reproduces none of them.

Why Cross-Industry Norms Don't Transfer

Three things move lift independently of creative quality. Channel changes the dose, since a single display impression and a 30-second video aren’t the same exposure. Brand size changes the headroom, since a well-known brand has little awareness left to gain.

And control-group method changes the estimate itself: a benchmark pooling randomized and self-selected studies mixes valid lifts with inflated ones, and the Gordon, Moakler and Zettelmeyer results suggest the inflation can be several-fold.

Build Your Own Benchmark

Your benchmark is your own prior lift studies, run on the same instrument, in the same channel, against the same audience definition. Each campaign measured the same way adds one reference point, and no source says how many points make the reference reliable.

Ask whether a lift is larger than the campaigns you’ve already measured, on the metric the campaign was built to move. This guide publishes no minimum lift that counts as good, because no source supports one.

Interpreting and Acting on the Result

Interpret a lift result by asking three questions in order. Is the lift distinguishable from zero? Is it large enough to matter for the decision? Does the pattern across metrics make sense for this campaign’s objective? A result that passes the first question and fails the other two isn’t a win.

A Significant Lift on the Wrong Metric

A campaign built to drive consideration that lifts only ad recall has been noticed without persuading anyone. Ad recall is closer to an exposure check than an effect measure, and a recall lift with flat consideration points to the creative or the offer, not the media plan.

A Non-Significant Lift

A non-significant result from an underpowered study isn’t evidence of no effect. Check the detectable-lift table before concluding anything. If the study could only detect a 12-point lift, a null result says the lift was probably smaller than 12 points and nothing more.

If one wave can’t settle it, a brand tracking series accumulates evidence across campaigns. Report the upper bound of the interval: “the campaign’s effect on consideration was at most six points” is an honest and useful sentence.

A Lift Larger Than Expected

Treat a surprisingly large lift as a prompt to check the design before celebrating. The usual causes are a control group that wasn’t truly eligible, differential completion between groups, and a survey environment that re-exposed one group.

A Worked Interpretation

A subscription app randomizes 20 percent of lapsed users into a holdout and sends a reactivation email to the rest, then invites the holdout and an equal-sized random subsample of the emailed group to the survey. It reaches 1,200 control and 1,180 exposed respondents, and consideration rises from 26.0 to 31.0 percent, a five-point absolute lift with an interval from 1.4 to 8.6 points.

In this illustrative example, ad recall rises by 14 points, and the control group’s recall of the email it never received sits at 9 percent. The false-recall item sits at 7 percent in both groups, so the recall figures are consistent with genuine memory plus a modest agreement baseline.

The email was noticed and moved consideration by somewhere between one and nine points. Whether that justifies the send depends on the threshold agreed before fielding, not on the verdict alone.

Turning the Result Into a Decision

Tie every lift result to the decision it was meant to inform. Common decisions include:

Extend the flight to more regions or a larger audience
Shift budget toward the channel or creative with the stronger lift
Revise the creative when recall lifts but consideration doesn’t
Retire the campaign when the interval excludes any meaningful lift

Agree the threshold that triggers each decision before fielding. Agreeing it afterward invites the result to choose the threshold.

Running the Lift Analysis With Claude or ChatGPT

A language model with code execution can compute lift for every metric, test it, and flag the defects that break a lift comparison, provided the prompt specifies the arithmetic instead of asking the question.

The prompts below cover what’s specific to lift studies. Cutting a result by segment is covered in the simple cross-tab guide, and auditing any AI-generated analysis is covered in the guide on analyzing survey data with AI.

What the Analysis Produces

A complete lift analysis produces one table per study: each metric’s control rate, exposed rate, absolute lift, 95 percent interval, z statistic, relative lift and headroom lift, plus the false-recall figures and a count of tests run.

A matched-market design adds the difference-in-differences estimate and a region-level check. You keep the design decisions the model can’t see: whether the holdout was random, whether it stayed suppressed, and whether the survey environment re-exposed anyone.

Prompt One: Lift per Metric With the False-Recall Baseline

What this prompt does: computes absolute, relative and headroom lift with a two-proportion test for every metric, reports false recall, and checks the groups for balance.

What it returns: a lift table saved as lift_table.csv, a balance check, and a mismatch log.

You’re analyzing an advertising effectiveness survey with an exposed group and a control group. The attached CSV is a survey export.

Use code to calculate this, not estimation.

Inputs:
- Group column: [GROUP COLUMN, default: customMetadata, key "arm"]
- Exposed value: [EXPOSED VALUE, default: exposed]
- Control value: [CONTROL VALUE, default: control]
- Metrics and their positive responses: [METRIC LIST, default: Q3 aided awareness = "Yes", Q4 ad recall = "Yes", Q7 favorability = top two boxes, Q8 consideration = "Yes"]
- False-recall question: [FALSE RECALL COLUMN, default: Q5_Response]
- Primary metric, named before fielding: [PRIMARY METRIC, default: Q8 consideration]
- Minimum valid responses per group and metric: [MIN BASE, default: 10]

Rules:
1. If the group value sits inside the customMetadata object, parse it first. If the group column is missing, or holds any value other than the two named, stop and report the unexpected values. Don’t guess.
2. For each metric, exclude blank, skipped and "Prefer not to say" responses from both numerator and denominator. Report how many were excluded per group.
3. Treat any group-metric cell with fewer than [MIN BASE] valid responses, including cells with zero responses, as under threshold. List every under-threshold cell by name and count, and don’t compute lift for it.
4. For each metric, compute the control rate, exposed rate, absolute lift in percentage points, a 95 percent confidence interval using the unpooled standard error, a z statistic using the pooled standard error, relative lift (absolute lift divided by control rate) and headroom lift (absolute lift divided by one minus control rate).
5. Report the false-recall item for both groups, and the control group’s rate on the real ad-recall question, as separate figures. Don’t subtract either from anything.
6. Count the metrics tested. Mark the primary metric. For every other metric, report whether it survives a Holm correction at 0.05 across the non-primary set.
7. Compare the groups on completion count and on every demographic column. Report any demographic share differing by more than [BALANCE FLAG, default: 5] percentage points.
8. If [WEIGHT VARIABLES, default: none] are named, weight the control group to the exposed group’s profile on those variables and report weighted and unweighted lift side by side.
9. Recompute by genuinely different methods. Rebuild each group’s counts from row-level 0 or 1 values with a separate crosstab instead of reusing the step 4 counts. Run a Pearson chi-square test without continuity correction and confirm it equals the square of the step 4 z. Bootstrap the absolute lift with [BOOTSTRAP DRAWS, default: 5000] resamples and report the percentile 95 percent interval beside the step 4 interval.
10. If any two methods disagree on a figure, or any count doesn’t reconcile with the file’s row total, list the mismatch in a section called Mismatches. Don’t silently correct it.

Save the lift table as lift_table.csv with one row per metric, and show it in the response.

The minimum base of 10 and the balance flag of 5 points are prompt settings that control when warnings print. They aren’t published minimums. The sample-size chapter decides how large each group should be.

Prompt Two: Difference in Differences for a Matched-Market Design

What this prompt does: computes the difference-in-differences lift for a matched-market pre-post study and checks it against a region-level read.

What it returns: did_table.csv with the respondent-level and region-level estimates, and a mismatch log.

You’re analyzing a matched-market advertising effectiveness survey. The attached CSV combines a before wave and an after wave.

Use code to calculate this, not estimation.

Inputs:
- Region column: [REGION COLUMN, default: Attributes_region]
- Exposed regions: [EXPOSED REGIONS, no default, so stop and ask if blank]
- Wave column and values: [WAVE COLUMN, default: customMetadata, key "wave", values before and after]
- Primary metric and positive response: [PRIMARY METRIC, default: Q3 aided awareness = "Yes"]
- Minimum valid responses per region and wave: [MIN BASE, default: 10]
- Region-count warning: [REGION FLAG, default: 5]

Rules:
1. Exclude blank, skipped and "Prefer not to say" responses on the primary metric from numerator and denominator, and report the exclusions per region and wave.
2. Treat any region-wave with fewer than [MIN BASE] valid responses, including region-waves with zero responses, as under threshold. List them and exclude them from both estimates.
3. Compute the positive rate for each of the four cells: exposed before, exposed after, control before, control after.
4. Compute the difference in differences in percentage points, with a standard error that combines the variance of all four cells. Report a 95 percent interval.
5. Compute a region-level estimate: for each region, the after rate minus the before rate. Compare the mean change in exposed regions with the mean change in control regions using a two-sample t test with the region as the unit.
6. Report the number of regions in each group. If either group has fewer than [REGION FLAG] regions, state that the respondent-level interval overstates precision.
7. Recompute the difference in differences with a linear probability model of the 0 or 1 response on exposed, after, and exposed times after, with standard errors clustered by region. Compare the point estimate with step 4, report both standard errors, and label the clustered one as the honest one when regions are few.
8. If any figure disagrees between methods, list it under Mismatches. Don’t silently correct it.

Save the results as did_table.csv and show it in the response.

The region warning of five is a prompt setting, not a published minimum. With very few regions the region-level test has little power, and the honest reading is often that the design can’t settle the question.

Setup and Data Preparation

Export the responses as CSV and confirm the group assignment is a column before opening the model. The export lists customMetadata for link parameters, Attributes_ columns for user attributes and Q#_Response columns for answers.

It lists no groups column, so stamp the group on every response at collection, either as a link parameter or as a user attribute. A file with no group column can’t be analyzed, and both prompts refuse to run without one. Confirm the exact column names in your export before running either prompt.

Turn on code execution in Claude or ChatGPT before starting, and attach the file before pasting the prompt so the data sits above the instructions.

Reading the Output

Read the Mismatches section first. A mismatch usually means a miscoded response, a duplicated row or a group label with a typo, and each of those changes the lift. Read the balance check second.

A demographic gap on an owned-channel study usually points to differential completion, and a gap on a platform study points to a targeting difference. Only then read the lift table, starting with the primary metric’s interval instead of its verdict.

What to Verify Before Reporting

Verify four things by hand before any lift figure reaches a slide:

Match the exposed and control row counts against the delivery log for each group
Recompute the primary metric’s rates in a spreadsheet from the raw file
Confirm the primary metric in the output matches the one named before fielding
Check that no under-threshold cell appears in the reported table

As of October 2026, we found no documented seed setting in either chat interface, so produce any figure you’ll report twice, in fresh conversations, and compare. Code execution reduces arithmetic error but doesn’t remove it, which is why the checks above stay in the workflow.

Sprig, and the constraint: Sprig MCP connects Claude and ChatGPT to study data directly, with access scoped to the authenticated user’s role, response data not used to train models, and an admin kill switch. Agents can’t launch or modify a live study.

The constraint for lift studies is size: MCP returns at most 1,000 responses per call (Sprig MCP documentation), so a powered two-group study needs several calls or a CSV export, and MCP hands data to the model instead of running the test.

Pitfalls When a Model Analyzes Lift

Seven failure modes recur, and six carry a published source.

Discovering themes is harder than applying them. Hill and colleagues, in PLOS Digital Health in April 2026, found models matched human analysts on coding against an existing codebook, 93.5 percent against 92.7, and were materially worse at inductive discovery.
Rare codes get over-predicted. Ashwin, Chhabra and Rao, in Sociological Methods & Research in May 2025, found non-random bias in 10 of 19 codes, with sparse codes over-predicted.
Outputs vary run to run. Thinking Machines Lab reported in September 2025 that 1,000 completions at temperature zero produced 80 unique outputs.
Information in the middle of a long context gets used less. Liu and colleagues, in Transactions of the Association for Computational Linguistics in 2024, documented this, which is why the data goes above the instructions.
Models agree with the user. Sharma and colleagues, at ICLR 2024, documented sycophancy, and OpenAI withdrew a model update in April 2025 for being overly agreeable. Never ask a model to confirm the lift you expect.
Verbatims contain personal information nobody asked for. The data-use terms that apply depend on the account tier, not the vendor.
Asking "did the campaign work" produces a winner every time. A file with two groups and six metrics holds enough comparisons that the model will find one, because it isn’t counting them. Specify the primary metric and the test, not the question.

The last pitfall is the multiple-metric problem in a new costume, and it’s the reason both prompts name the primary metric and the test.

The Critique You Should Know

The strongest critique of survey-based ad effectiveness measurement is that most studies can’t be trusted, and it came from inside the industry. The strongest defense is that randomization fixes most of what the critique identified. Both positions are sourced, and anyone presenting lift results to a skeptical finance team should know each.

The Case Against: Low Response and Non-Equivalent Groups

Lavrakas, in a 2010 white paper commissioned by the IAB, reviewed the methods behind internet ad effectiveness studies and concluded: "one cannot be confident whether the findings of most IAE studies are right or wrong" (IAB white paper, 2010, retrieved October 15, 2026). Two defects drove the verdict.

Response rates were "less than 5% and in many cases are far less than 1%," and the comparison groups in "many quasi-experimental IAE studies often are not part of the target population of interest."

He added that studies "may be overestimating the positive effects of the internet ad campaigns," but that "there presently exists no valid empirical basis on which to estimate by exactly how much."

The Case Against: Observational Comparisons Overstate Lift

Gordon, Zettelmeyer, Bhargava and Chapsky (2019), reporting 15 United States experiments in Marketing Science, found that in half of the studies the estimated percentage increase in purchase outcomes was off by a factor of three across all methods.

Gordon, Moakler and Zettelmeyer (2023) extended the test to 663 experiments and concluded that even with over 5,000 user-level features, non-experimental methods couldn’t reliably recover the experimental result. Neither paper studies survey outcomes directly. Both study the same selection problem a self-reported exposed group suffers from, with far better data than any survey carries.

The Case Against: Conventions That Invite False Positives

Google reports Brand Lift results at "80% 2-sided confidence intervals," describes certainty of lift as "1 - p-value," and states that advertisers "can receive valuable and directional insights from your brand lift results at ≥50% certainty of lift." That’s a disclosed convention, not a hidden one (Google Ads Help, retrieved October 15, 2026).

But at an 80 percent interval, a metric with no true effect shows a significant difference one time in five, and across six metrics the chance of at least one false lift is 73.8 percent. Comparing a platform result with your own 95 percent test means comparing two different standards.

The Counter-Critique: Randomization Answers Most of It

Lavrakas’s central threats were non-equivalent comparison groups and fragile cookie-based control assignment. Randomized delivery holdouts, such as the ghost ads design in Johnson, Lewis and Nubbemeyer (2017), address both at the point of delivery. His nonresponse concern survives randomization, because both groups still respond at low rates. His selection concern largely doesn’t.

Goldfarb and Tucker (2011) used "a large randomized field experiment on 2,892 distinct Web advertising campaigns," with half of respondents "randomly exposed to the ad and half ... not exposed to the ad but did visit the website." The design was sound enough to publish findings on targeting and obtrusiveness in Marketing Science.

A survey lift study with a random control group is a legitimate experiment, and the method’s poor reputation comes mostly from studies without one.

The Counter-Critique: Self-Reported Recall Tracks Real Exposure

Romberg and colleagues (2020), in the International Journal of Environmental Research and Public Health, compared 458 people with tracked digital exposure to a campaign against 506 matched controls.

Exposed participants had significantly higher odds of reporting recall, and "each additional ad exposure increased the odds of self-reporting higher exposure frequency by 8%." Niederdeppe (2005), in the Journal of Health Communication, found aided and confirmed recall both valid exposure measures for a state tobacco campaign. Both studies evaluate public-health campaigns, so their transfer to commercial advertising is plausible, not proven.

The Position This Guide Takes

A lift survey is a valid measure of what a campaign did to memory and attitude when the control group is random and the study is powered. It isn’t a sales measure, and it isn’t valid at all when the control group is self-selected.

Most of the method’s bad reputation belongs to the second case. Consideration and awareness also earn their place as leading indicators inside market-response models (Srinivasan, Vanhuele and Pauwels, 2010; Hanssens and colleagues, 2014), which the brand tracking guide covers in detail.

Common Mistakes in Advertising Effectiveness Surveys

Nine mistakes recur in the lift studies this guide describes. Each passes a casual review, which is why each needs a named check.

Comparing the Ad-Aware With the Unaware

The most frequent mistake is defining the exposed group by who says they saw the ad. Brand users notice a brand’s advertising more, so the comparison measures prior usage. Randomize before launch instead.

Letting the Holdout Leak

A holdout that receives the campaign through retargeting, a second email stream or a partner channel stops being a control. Suppress the holdout list on every channel for the full flight, and audit the suppression in the delivery logs. Partner channels and affiliate networks are the leaks most often missed, because they run on lists the marketing team doesn’t manage directly.

Surveying the Groups Differently

Fielding the control group from a research panel and the exposed group from the campaign’s own audience changes the population, the incentive and the environment at once. A difference in any one of the three is enough to move the lift. Field both groups from the same source in the same window.

Reading Every Metric That Crosses a Threshold

Six metrics tested separately produce frequent false lifts. Name one primary metric before fielding, and apply a correction to the rest.

Declaring No Effect From an Underpowered Study

A null result from an underpowered study says only that the lift was probably smaller than the study could detect. Read that detectable lift from the detectable-lift table, and report the upper bound of the interval instead of the word "none."

Restricting the Exposed Group to Openers or Clickers

On email, push and in-app channels, assignment isn’t exposure. Dropping unopened sends from the exposed group keeps only the most engaged people, which reintroduces the self-selection that randomization removed. Compare the full assigned groups and accept that unopened sends dilute the lift.

Mixing an 80 Percent Standard With a 95 Percent One

A platform result reported at an 80 percent interval and an own survey tested at 95 percent aren’t the same standard. Restate both at the same confidence level before comparing, or report each with its level beside it.

Surveying on the First Day of the Flight

A survey fielded as the campaign launches measures who has been reached, not what the campaign did. Wait until the exposed group has reached the media plan’s planned frequency.

Ignoring the False-Recall Baseline

Reporting the exposed group’s ad recall without the control group’s recall of the same ad overstates memory. The control group never saw the ad, so its rate is the share of agreement the question produces on its own, and it belongs beside the exposed figure in every report.

Holdout leakage and differential fielding are the two easiest mistakes to miss in review, because the delivery logs often go unchecked and each group looks healthy on its own.

Synthetic Respondents and Advertising Effectiveness

Synthetic respondents can’t measure advertising effectiveness, for any group or any metric. The method measures what exposure did to a real person, and a synthetic respondent was never exposed. The broader evidence points the same way.

Bisbee and colleagues (2024), in Political Analysis, found synthetic averages that corresponded closely to real ones, but 48 percent of regression coefficients differed significantly, and 32 percent of those flipped sign. A lift estimate is a difference between two groups, the kind of estimate that study found synthetic data distorts. Withhold the campaign from a real group instead of simulating one.

Synthetic Respondents Aren't Synthetic Control

The term synthetic control refers to something different and legitimate. Abadie, Diamond and Hainmueller (2010), in the Journal of the American Statistical Association, introduced a method that builds a weighted combination of untreated regions to stand in for a treated one. It suits campaigns run in a single region, and it uses real outcomes throughout, which is the difference that matters.

Synthetic respondents can help draft and pressure-test an instrument before fielding, such as checking that a screener doesn’t name the brand. They can’t supply either group of the comparison.

How Sprig Supports an Advertising Effectiveness Survey

Sprig supports the owned-channel and matched-market branches of this method, and it doesn’t support the delivery-layer branch. It can’t observe ad exposure, it ships no lift calculation or significance test for proportions, and it publishes no weighting method.

What the Platform Covers

The strengths map onto the fielding half of a lift study. CSV groups hold the exposed and holdout lists, link parameters stamp the group on each response, and user attributes arrive through the SDK, the API or a CSV.

Panels with location filters, email, link, in-product and mobile delivery let one instrument reach both groups of an owned-channel study, and both region sets of a matched-market study, without a second survey vendor.

An Owned-Channel Lift Study, Step by Step

The owned-channel branch runs in six steps:

Export the eligible list from the CRM or messaging tool and randomize the holdout
Upload the exposed and holdout lists as two groups with an arm column, or set an arm attribute on each user
Build one survey from the frozen questionnaire and target it at both groups
Stamp the group on each response through a link parameter or the user attribute
Field both groups in the same window once the campaign reaches its planned frequency
Export the responses as CSV and run the lift prompt above

Step 1 happens outside the platform, and it’s the step that makes the study valid. Every later step only preserves what the randomization created.

Where the Platform Stops

Several documented absences shape how a lift study runs here. There’s no impression tracking or ad-platform integration, so the assignment comes from outside. Sprig’s Analyze Agent lists crosstabs, t-tests, ANOVA and correlation but is labeled coming soon, with a preview banner, as of October 2026, and no listed test is named as a two-proportion test, so lift analysis runs in the prompts above or a spreadsheet.

Quotas are Enterprise-only and count only respondents who finish. Panels are Enterprise-only and link-only, and Sprig hosts data on AWS in the United States with no data-residency options we could find in its public documentation, which matters for a multi-country campaign.

For a lift study with a real budget behind it, the market and consumer insights solution is where this workflow lives.

Alternatives and Adjacent Methods

Choose the method by the question the decision turns on. A lift survey answers whether one campaign moved an attitude, and five adjacent methods answer the questions it can’t:

Message testing: ranks creative before launch under forced exposure
MaxDiff: ranks many claims when a stable order matters more than a level
Brand tracking: detects drift over time on a fixed instrument
Conversion lift tests: measure behavior on the full randomized population
Marketing mix modeling: connects spend across channels to revenue

Message testing and lift studies get confused, and the difference is timing. Message testing screens before money is spent, while a lift study measures after it’s spent.

A geo experiment with sales as the outcome is the neighbor most often confused with a matched-market lift study. Both withhold the campaign from comparable regions, but the geo experiment reads logged sales instead of survey answers, so it answers the revenue question a survey can’t.

Norm-based advertising pre-testing is a separate job again. Specialist platforms that compare a creative score against category norms suit teams that need a normed score, and the AI market research tools roundup covers them.

If the same people also need a view of attitudes beyond one campaign, a usage and attitudes study is the closer fit. Depth interviews, moderated or AI-moderated, answer why a campaign failed to persuade from a few participants, and they complement a lift study without replacing it.

Frequently Asked Questions

What Is an Advertising Effectiveness Survey?

An advertising effectiveness survey compares brand metrics between people exposed to a campaign and a control group who were eligible but not exposed. The difference between the groups is the lift, and it’s valid when the control group was formed at random.

What Is a Brand Lift Study?

A brand lift study is another name for an advertising effectiveness survey. It measures the change a campaign produced in awareness, ad recall, favorability, consideration or intent by comparing an exposed group with an unexposed control group surveyed in the same window.

How Many Respondents Do I Need for a Brand Lift Study?

A brand lift study needs enough respondents per cell, meaning per exposed group and per control group, to detect the smallest lift worth acting on. At a 30 percent baseline, detecting a five-point lift takes about 1,377 per cell at 95 percent confidence and 80 percent power, and detecting a three-point lift takes about 3,800.

Is 200 Respondents per Cell Enough for a Lift Study?

Two hundred respondents per cell isn’t enough for most lift studies. At a 30 percent baseline it detects only true lifts of about 13.5 points, while published lifts on favorability and intent run far smaller. The 200 figure traces to 2011 IAB guidance and carries no power calculation.

How Do You Create a Control Group for an Ad Campaign?

Create a control group by randomly withholding the campaign from part of the eligible audience before launch. On owned channels, randomize the list and suppress the holdout everywhere. On paid platforms, use the platform’s randomized lift study. Where neither works, hold out comparable regions.

What Is the Difference Between Brand Lift and Conversion Lift?

Brand lift measures a change in attitude or memory through a survey, while conversion lift measures a change in behavior, such as purchases or sign-ups, from logged outcomes. Both need a randomized control group, and conversion lift is the right instrument when the decision turns on revenue.

Can You Measure Ad Effectiveness Without a Control Group?

Ad effectiveness can’t be measured without a control group of some kind. A before-and-after comparison detects that a metric moved but can’t separate the campaign from seasonality, competitor activity or a category shock. A matched set of untreated regions is the minimum substitute when no randomized holdout exists.

Does Ad Recall Predict Sales?

Ad recall is closer to an exposure check than a sales predictor. Published work shows self-reported recall tracks actual exposure, but consideration and awareness carry stronger evidence as leading indicators of sales inside market-response models (Srinivasan, Vanhuele and Pauwels, 2010). Treat recall as a check that the campaign was noticed.

What Is the Difference Between a Brand Lift Study and a Brand Tracking Survey?

A brand lift study compares an exposed group with a control group to attribute a movement to one campaign, while a brand tracking survey measures the same metrics at fixed intervals to detect drift. Brand tracking detects change, and a lift study attributes it.

What Questions Should an Advertising Effectiveness Survey Ask?

An advertising effectiveness survey should run from least to most prompted: a screener that never names the brand, then unaided recall, then aided awareness and ad recall with a false-recall check, then favorability, consideration and intent, and demographics last. The frozen questionnaire above lists all 11 items in order.

How Long Should an Advertising Effectiveness Survey Be?

An advertising effectiveness survey should be as short as the primary metric allows. The questionnaire in this guide uses 11 items. Longer instruments raise break-off, and break-off that differs between the exposed and control groups changes who is being compared.

How Big Should the Holdout Be?

A holdout needs to be large enough to return the control group’s required completes, so work backward from the cell size. Divide the completes needed per cell by your expected response rate. At 1,377 completes and an illustrative 10 percent response rate, each group needs about 13,770 invitations, and the rest of the audience can receive the campaign.

How Do You Run a Lift Study Across Several Countries?

Run a lift study across countries as a separate randomized comparison in each market, with its own holdout and its own cell-size calculation. Translate the questionnaire once, freeze it, and report each market’s lift before any pooled figure, because a pooled lift weighted by response instead of audience size misstates the campaign.

When Should I Field a Lift Survey?

Field a lift survey after the campaign has reached its planned frequency and before the flight ends, with an optional second wave a few weeks after. Field both groups over the same days and for the same length of time.

Can a Small Audience Run a Lift Study?

A small audience can run a lift study only for large effects. At 200 respondents per cell and a 30 percent baseline, the study detects a true lift of about 13.5 points, so smaller true effects go unread. A small audience is better served by a pre-post tracker with a holdout, or by pooling several waves.

What Is a Good Brand Lift?

There’s no independent cross-industry benchmark for a good brand lift. The best benchmark is your own prior lift studies on the same instrument, channel and audience. The largest disclosed commercial dataset reports that favorability moves about three points after a typical campaign.