It's the most practical way for most businesses to establish causation rather than correlation. Attribution can only observe which touchpoints preceded a sale; withholding the advertising from a comparable population is what turns that observation into evidence.
The design requirements are strict and frequently ignored. Test and control regions must track each other historically — twelve months of correlated revenue is the usual bar — and the withholding must be total rather than reduced, because partial treatment produces a signal that can't be cleanly interpreted.
Crucially, the measurement uses total revenue in each region from your own systems, not platform-attributed revenue. Using attributed data reintroduces exactly the bias the test exists to eliminate.
Why Geo Holdout Test matters
It's the only method that reliably distinguishes advertising that creates demand from advertising that harvests it. Results routinely contradict attribution dashboards, and the channels with the highest attributed return are frequently the least incremental.
A workable design
Split comparable metro areas into test and control, verify twelve months of correlated revenue trends, then pause the channel entirely in control for four to six weeks. Compare total revenue, normalized for the historical relationship between the groups. Watch control for a fortnight afterwards — a rebound means demand was delayed rather than lost, which changes what the result means.
Common mistakes
Running it for two weeks
Too short to cover multiple purchase cycles or let delayed conversions land. Four to six weeks minimum, longer for considered purchases.
Choosing markets that don't match
Test and control need similar seasonality, demographics and baseline performance. Mismatched regions produce a confident answer that happens to be wrong.
Measuring attributed rather than total revenue
Defeats the purpose entirely. The whole point is to sidestep attribution.
Testing during an unusual period
A holdout spanning a major promotion, a product launch or a seasonal peak measures the event rather than the channel.
Where we work on this