How long should an incrementality test run?

There is no standard duration. It depends on baseline conversion volume, the size of effect you want to detect and how long customers take to convert. Meta's GeoLift documentation recommends covering at least one full purchase cycle and a minimum of 15 days of daily data (4 to 6 weeks if the data is weekly), and running a power analysis before launch to find out how long a test needs; platform lift tools set their own requirements. Low-volume accounts may need a longer test or a larger holdout to separate a real effect from normal variation.

What's the difference between incrementality testing and A/B testing?

Both are controlled experiments, and they answer different questions. An A/B test usually compares two versions of something that both groups still receive, such as two creatives or two landing pages. An incrementality test compares exposure with no exposure to estimate what the advertising itself added to an outcome you measure, such as conversions or sales.

Can incrementality testing replace platform attribution?

Not entirely. Platform attribution is still useful for day-to-day decisions like pausing underperforming ad sets. Incrementality testing is better suited to larger questions like channel budget allocation. How often to repeat a test depends on how much your spend, creative, audiences and seasonality change; there is no standard cadence.

Does incrementality testing work for small budgets?

It is harder. Small accounts often have too few conversions to detect an effect within a reasonable window, and a test that is too small can miss a real effect, so a null result from an underpowered test is not evidence that advertising does nothing. A power calculation before launch shows whether your volume is enough; if it is not, options include a longer test, a larger holdout or a different method, and very low-spend accounts may have to rely on directional signals instead. Geo-level tests do not remove this problem: Meta recommends people-based tests over geo-based ones where possible because they have higher statistical power.

Definition

Incrementality Testing: Definition & How It Works

Incrementality testing isolates how much of a reported conversion an ad channel actually caused, using a holdout group as the baseline for comparison.

The OpenAdLibrary TeamAd intelligence & native advertising research

September 5, 2026 · Updated October 25, 2026 · 2 min read

Editorial illustration: Incrementality Testing: Definition & How It Works

Incrementality testing is a controlled experiment that estimates how much of the conversion activity reported for an ad channel the advertising actually caused, rather than relying on the conversions the platform's tracking attributes to its ads. You hold back a group of users or geographies from seeing a specific channel's ads, then compare the outcome against a group that saw them. A well-designed test reads the gap between the groups as an estimate of the channel's incremental effect for that audience, period and setup, reported with its uncertainty; the outcomes that remain in the holdout group are the ones that happened without the ads.

Why platform-reported numbers can differ from incremental results#

Ad platforms attribute conversions to ads using their own rules. If someone was already going to buy through organic search or a branded search click, and also happened to see a retargeting ad along the way, standard conversion tracking can still credit that ad inside its attribution window. Softer credit, where a platform counts a conversion after someone merely saw an ad, adds to the gap; that mechanic is called view-through attribution. The conversions a platform reports are therefore not the same thing as the conversions its ads caused. Incrementality testing asks a different question: if this channel had not advertised, would the conversion still have happened?

That is a different question from the one a tracking pixel answers on its own, so incrementality testing is used alongside platform dashboards, not instead of them.

How teams run it in practice#

Several designs are used. A geo-level test assigns regions to a test or holdout condition, for example by running a channel's ads in some regions and withholding them in others, and compares outcomes such as sales or conversions between the regions; Google's geo experiments randomly assign non-overlapping regions, while Meta's GeoLift builds a synthetic control from untreated regions. A user-level test uses an ad platform's conversion-lift tooling: Meta's Conversion Lift randomly assigns people to a test group that sees the ads and a control group that does not, and Google Ads Conversion Lift offers user-based and geography-based studies, though Google says it is not available for all accounts.

How the effect is estimated depends on the design. In a randomized user-level test, the estimate is the difference in outcomes between the test group and the holdout group, which can be reported as extra conversions or as a percentage over the holdout. As a hypothetical example, if 10,000 test users produce 300 conversions and 10,000 holdout users produce 250, the estimated lift is 50 conversions, or 20% over the holdout, even if the platform attributed all 300 conversions to its ads. Geo tests usually do not rely on that simple subtraction: they compare treated regions with what a model, such as a synthetic control built from untreated regions, says they would have produced without the ads. A channel with a strong platform-reported ROAS can show a much smaller incremental effect in a test; how large the gap is depends on the channel, audience and measurement window.

What it's used for#

Media buyers use incrementality tests to inform budget allocation across networks, not just to audit one channel in isolation. A result describes the audience, period and spend level that were tested.

Public ad archives cannot measure incrementality. An ad that was observed for a long time does not show that it caused additional conversions, because ad longevity records how long an ad was seen, not spend or results. What the archives can offer is hypotheses: OpenAdLibrary's ad intelligence pages show which creatives and networks an advertiser has been observed using, and for how long, which can help you decide what to test with your own holdout before you commit test ad spend.

Common mistakes#

The holdout group needs to be large enough, and assigned randomly or matched carefully enough, to detect a real signal and not just noise. A short test on a low-volume channel may not have enough conversions to detect an effect, and Meta's GeoLift documentation warns that running a geo test without a prior power analysis leads to a high chance of failing to find lift even when it exists. A seasonal spike or a site-wide promotion does not by itself invalidate a randomized test, because both groups experience it, but the measured lift then describes that period and may not hold in normal weeks. An event that reaches only one group, such as a regional promotion in a geo test, can bias the result, so keep other marketing constant across test and control regions. Run the test during a representative period that covers at least one purchase cycle, size the holdout with a power calculation, and don't stop early just because the interim number looks good, or bad.