One answers what happened. One answers what caused it.
These are not competing methods for the same job. They answer different questions, and most measurement arguments are really two people answering different ones at each other.
Our bias, declared
Use attribution to allocate inside a channel and incrementality to decide whether a channel deserves budget at all. Attribution is cheap, continuous and observational — it tells you which touchpoints preceded conversions, which is genuinely useful for tuning campaigns against each other. It cannot tell you what would have happened without the touchpoint, and no model sophistication changes that, because the counterfactual was never observed. Incrementality is expensive, periodic and causal. Running only the first is the default failure; running only the second is impractical.
Side by side
| Factor | Attribution | Incrementality testing |
|---|---|---|
| Question it answers | Which touchpoints preceded the sale | What the channel actually caused |
| Cost to run | Effectively free, continuous | Real revenue withheld |
| Speed of feedback | Daily | Weeks to a quarter |
| Granularity | Campaign, ad set, keyword | Channel, at best |
| Establishes causation | No | Yes |
| Survives brand-search bias | No — over-credits it heavily | Yes |
| Works with degraded tracking | Increasingly poorly | Unaffected — measures totals |
| Usable for daily bidding | Yes | No |
| Defensible to a CFO | Contested | Hard to argue with |
| Requires separable markets | No | Yes — a real constraint |
Choose attribution when
You are allocating inside one channel
Given a fixed budget, which campaigns and keywords get it. Attribution does this well and a holdout cannot do it at all — you are not going to run a geo test per ad set.
You need a signal to optimise against daily
Bidding algorithms need conversions fed back continuously. That is attribution's job and there is no substitute, which is why the answer is never to abandon it.
The decision is small and reversible
Most day-to-day media decisions do not justify the cost of causal measurement. Reserve that for the allocations large enough to matter.
Choose incrementality testing when
You are about to move or cut a large budget
Any decision big enough to change the shape of the business deserves a causal answer, because attribution's bias is systematic rather than random and points the same way every time.
The channel looks suspiciously good
Branded search, retargeting and email to engaged lists report returns that would be extraordinary if real. These are exactly where attribution over-credits, and where a holdout most often changes minds.
Tracking has degraded and you know it
Holdouts compare total revenue between regions. They do not care about cookies, consent rates or cross-device paths, which makes them more robust precisely as attribution gets less reliable.
Marketing and finance disagree about what worked
This is the argument attribution cannot settle, because both sides can produce a defensible model. A holdout produces a number neither side chose the methodology for.
What each one actually costs
Attribution's cost is not the tooling, which is usually already paid for. It is the decisions made on it — and specifically the systematic misallocation caused by its bias toward demand capture. That cost is invisible, recurring and frequently much larger than any measurement budget.
Incrementality's cost is explicit and uncomfortable: you withhold advertising from a set of markets and lose the revenue it would have produced. The useful framing is that this cost is proportional to how well the channel works — a test on a channel doing nothing is nearly free, and a test on a channel doing a great deal is expensive precisely because you have found something valuable.
The mistake worth avoiding is treating this as a budget line at all. One holdout a year on your largest channel is small against the allocation it informs, and it makes every attribution number you read for the following twelve months more interpretable — because you now know roughly how far off the model is and in which direction.
Related questions
No, and the reason is structural rather than technical. Every attribution model — last-click, data-driven, position-based — distributes credit among touchpoints that were observed. None of them can observe what would have happened if the touchpoint had not existed, because that world does not exist in the data. Sophistication improves how credit is divided; it cannot manufacture a counterfactual.
Other comparisons
Services referenced
Get the audit before you decide anything.
Thirty minutes, free, no obligation. We'll tell you honestly whether you need an agency at all — including when the answer is that you don't.