Measure the lift a channel actually adds by running a holdout, not the correlation between exposure and conversion the channel's own report shows
Every paid channel reports a conversion number, and every number is too high. Attribution windows count conversions the channel did not cause, retargeting takes credit for customers who would have come back anyway, and view-through counts the brand-aware customer who saw an ad and converted for unrelated reasons. The self-reported channel performance is not a lie; it is a measurement of correlation between exposure and conversion that does not isolate the causal lift the channel added. Budget decisions made on those numbers over-invest in the channels best at taking credit and under-invest in the channels doing the work nobody can see.
Incrementality testing is the answer the industry has been pointing to for a decade and most teams still do not run, because the operational discipline (suppressing a campaign in part of the country or part of the audience, holding the holdout long enough for the test to power, accepting the short-term revenue cost of the suppression) is genuinely hard. The test designs that work (geo-split, audience holdout, ghost-bid) all share the structure: an exposed group that gets the channel, a holdout group that does not, and a measurement of the conversion gap that the channel can actually claim caused.
This recipe builds incrementality testing as a measurement habit, not a one-off study, and uses the results to calibrate the everyday attribution the team runs the rest of the year on.
A measured incremental lift per channel that the team trusts to size budget against, with the confidence interval that says how strong the read is. The metric is the channel's incremental conversion lift with its confidence bound, measured against a holdout that did not see the channel. The value shows up in budget decisions that move money from channels with low measured lift to channels with high measured lift, and in the calibration of the everyday attribution that runs without a holdout (the team multiplies the self-reported numbers by the measured lift coefficient until the next test refreshes it).
The second outcome is the discipline of running incrementality continuously on the channels most worth measuring, rather than as a quarterly exercise that produces a number nobody acts on by the time the next test is designed.
A warehouse with the conversion data joinable to the test assignment, an analytics environment that can do the lift estimation (matched-pair t-test for simple geo designs, difference-in-differences regression for richer ones), and the operational capability to suppress the channel cleanly in the holdout (a region the platform can exclude, an audience the campaign can hold out, a campaign that can be paused on a schedule). Composable stacks suit this because the test design and the lift estimation both live in the warehouse and the platform-side suppression is configuration. The capability that decides whether the test produces a usable read is the test design: a power calculation that says the test needs eight weeks to detect a five-percent lift means a test run for four weeks tells the team nothing, and a test run on a holdout that contaminates (people in the exposed and holdout geos see each other's ads, or audiences overlap) produces a lift estimate compressed toward zero.
Compare the tools on Martech Stack Builder
Analytics owns the test design, the lift estimation, and the read of whether the result is significant. Data science owns the power calculation, the regression that controls for confounders, and the confidence intervals that turn the lift estimate into a number the team can act on. Marketing ops owns the operational side: the campaign suppression in the holdout, the spend reallocation the test creates, and the discipline of holding the holdout long enough rather than ending early when the suppressed region underperforms. Legal advises where the consent and disclosure picture is non-trivial, because a holdout that withholds an experience the customer would have expected can be a disclosure question in stricter jurisdictions. The recipe takes months because the test takes months. A power calculation that says four weeks is enough is rare for the lifts most teams are trying to measure, and rushing the test to a six-week timeline because it has to fit a quarter is the most common reason incrementality studies produce non-significant results and the team gives up on the method.
Work in this order. Power the test before it starts, and hold it for the full duration.
Incrementality testing as default measurement covers steps 1 to 3 and 7, consent-scoped signal collection step 4, signal quality monitoring steps 5 and 6, and closed-loop attribution with consent-aware fallback step 8.
The first failure is the test that ends early. A geo-split test where the suppressed region underperforms in week two creates real revenue pressure, and ending the test before it powers preserves the spend but produces a non-significant lift estimate, which gets read as "the channel does not work" when really it means "we did not test long enough to tell." Holding the test for the duration the power calculation specifies is the discipline that makes the result act-on-able.
The second is the contaminated holdout. A holdout audience that overlaps with the exposed audience (a Facebook holdout that still sees the brand's Google ads, a geo holdout where the channel's targeting bleeds across state lines, an email holdout that still receives lifecycle messages) compresses the lift estimate toward zero, and the team concludes the channel underperforms when really the test never measured it cleanly. A clean holdout structure is what makes the lift number trustworthy.
The third is treating the test result as permanent. A measured lift in Q1 informs budget through Q2, but channel performance changes (creative, audience, seasonality, competitive context), and a six-month-old incrementality estimate has the same problem as a six-month-old MMM run. Incrementality has to be a habit, refreshed on the channels that matter most, or it becomes a one-off the budget process drifts away from.
The Workshop works out with your team which of these matter for your stack right now, and what to do first: a 90-minute session with the people who own the decision.
The geo-split or audience-holdout structure that fits each channel, the power calculation that says how long the test runs, the contamination checks that keep the result valid, and the calibration loop that uses the incrementality result to weight the everyday attribution: those are the decisions that turn channel reports from correlation into measured causation the team will move budget on.
Did this recipe match your situation?Anonymous response. Sign up to leave a longer note tied to your account.
Pick an attribution model, lock it, document why, and reconcile its outputs against incrementality, so the channel report becomes one rule the team can defend rather than a debate per meeting
Group users by when they signed up or first acted, then watch each group's engagement curve, so a quiet quarter doesn't disguise a real shift in who is sticking
Watch the conversion events at the resolution that catches a deploy breaking the cart-success fire before reporting starts surprising people, and surface the fix as an operational alert rather than a quarterly investigation