One answers what happened. One answers what caused it.
These aren't competing methods for the same job. They answer different questions, and most measurement arguments are really two people answering different ones at each other.
Our bias, declared
Use attribution to allocate inside a channel and incrementality to decide whether a channel deserves budget at all. Attribution is cheap, continuous and observational. It tells you which touchpoints came before conversions, which is useful for tuning campaigns against each other. It can't tell you what would have happened without the touchpoint, and no model sophistication changes that, because the counterfactual was never observed. Incrementality is expensive, periodic and causal. Running only the first is the default failure. Running only the second is impractical.
Side by side
| Factor | Attribution | Incrementality testing |
|---|---|---|
| Question it answers | Which touchpoints preceded the sale | What the channel actually caused |
| Cost to run | Effectively free, continuous | Real revenue withheld |
| Speed of feedback | Daily | Weeks to a quarter |
| Granularity | Campaign, ad set, keyword | Channel, at best |
| Establishes causation | No | Yes |
| Survives brand-search bias | No, over-credits it heavily | Yes |
| Works with degraded tracking | Increasingly poorly | Unaffected, measures totals |
| Usable for daily bidding | Yes | No |
| Defensible to a CFO | Contested | Hard to argue with |
| Requires separable markets | No | Yes, a real constraint |
Choose attribution when
You're allocating inside one channel
Given a fixed budget, which campaigns and keywords get it. Attribution does this well and a holdout can't do it at all. You aren't going to run a geo test per ad set.
You need a signal to optimize against daily
Bidding algorithms need conversions fed back continuously. That's attribution's job and there is no substitute, which is why the answer is never to abandon it.
The decision is small and reversible
Most day-to-day media decisions don't justify the cost of causal measurement. Reserve that for the allocations large enough to matter.
Choose incrementality testing when
You're about to move or cut a large budget
Any decision big enough to change the shape of the business deserves a causal answer, because attribution's bias is systematic, not random and points the same way every time.
The channel looks suspiciously good
Branded search, retargeting and email to engaged lists report returns that would be extraordinary if real. These are exactly where attribution over-credits, and where a holdout most often changes minds.
Tracking has degraded and you know it
Holdouts compare total revenue between regions. They don't care about cookies, consent rates or cross-device paths, which makes them more robust precisely as attribution gets less reliable.
Marketing and finance disagree about what worked
This is the argument attribution can't settle, because both sides can produce a defensible model. A holdout produces a number neither side chose the methodology for.
What each one actually costs
Attribution's cost isn't the tooling, which is usually already paid for. It's the decisions made on it, and specifically the systematic misallocation caused by its bias toward demand capture. That cost is invisible, recurring and frequently much larger than any measurement budget.
Incrementality's cost is explicit and uncomfortable: you withhold advertising from a set of markets and lose the revenue it would have produced. The useful framing is that this cost is proportional to how well the channel works. A test on a channel doing nothing is nearly free, and a test on a channel doing a great deal is expensive precisely because you have found something valuable.
The mistake worth avoiding is treating this as a budget line at all. One holdout a year on your largest channel is small against the allocation it informs, and it makes every attribution number you read for the next twelve months easier to read, because you now know roughly how far off the model is and in which direction.
Related questions
No, and the reason is structural, not technical. Every attribution model (last-click, data-driven, position-based) distributes credit among touchpoints that were observed. None of them can observe what would have happened if the touchpoint had not existed, because that world doesn't exist in the data. Sophistication improves how credit is divided. It can't invent what would have happened otherwise.
Other comparisons
Services referenced
Get the audit before you decide anything.
Thirty minutes, free, no obligation. We'll tell you honestly whether you need an agency at all, including when the answer is that you don't.