How can I tell if a channel is creating revenue or just capturing revenue I'd have gotten anyway?

Attribution is bookkeeping, not an experiment

An attribution model assigns credit for revenue that happened. It has no mechanism for knowing what would have happened otherwise. Those are different questions, and the second one is the one that determines whether spend is worth continuing.

This distinction gets ignored because attributed revenue is available daily and incrementality is not. But a channel can post excellent attributed revenue and be entirely redundant, and no amount of refining the model will reveal it.

The channels most likely to be capturing rather than creating

• Brand search. Someone typing your company name has already decided. Some of that traffic would arrive organically. Some genuinely would not, especially where competitors bid on your name.
• Retargeting. By construction it only reaches people who already engaged.
• Map and listings clicks from customers who already know you. Real revenue, ambiguous causation. This is one reason local listings data is better read as a demand signal than as a channel scorecard.

Three tests a service business can actually run

Geographic hold-out is the cleanest. Turn a channel off in one comparable service area, leave it on elsewhere, and compare revenue trajectories against each area's own history. Zip-level or DMA-level splits work because service businesses are geographically bounded in a way that most advertisers are not.

Time-based on and off cycles are second best. They are confounded by seasonality and weather, so they need multiple cycles and a control channel. Third, and weakest but always available: plot spend changes against total company revenue over a long window and look for whether the total responds at all. A channel whose spend doubles with no movement in the company total is a candidate for a real test. That kind of correlation work is what an AI business analyst layer is good at surfacing for a human to then test properly.

Be honest about what a test costs

A hold-out means deliberately giving up some revenue for a period, in a market you care about. That is a real cost and the reason most of these tests never get run. The counterargument is that the alternative is funding redundant spend indefinitely.

Run them rarely, on the largest line items, with a defined duration decided in advance. And accept the result even when it is inconvenient, which is the part that actually requires discipline.