How to measure incrementality in retail media (and why most platforms get it wrong)

Your retail media campaign comes back with a 6:1 ROAS. Good number. Then your finance director asks the obvious question: did the spend actually drive those sales, or would most of them have happened anyway?

That's an incrementality question, and it's the one most retail media reports aren't designed to answer properly.

TL;DR Reported ROAS counts every sale that happened during a campaign. Incrementality measures only the sales the campaign actually caused. The gap between the two can be significant, and closing it requires a proper test setup, not a better-looking dashboard. Here's what to ask for.

What is incrementality, and why doesn't standard ROAS measure it?

Incrementality is the share of your sales that genuinely came from the ad spend, rather than sales that would have happened regardless. A campaign with a strong ROAS isn't necessarily incremental, and a campaign with a modest ROAS might be highly incremental, because the two numbers measure different things and only one of them tells you whether the budget is working.

The Advertising Research Foundation's 2025 study on retail media networks found that while metrics like CTR, sales and conversion rates dominate campaign evaluation, advanced experimental methods for measuring incrementality, such as synthetic control and matched market tests, remain among the least utilised approaches in the industry. The IAB and IAB Europe put it plainly in their joint guidelines on incremental measurement: "Incrementality differs from attribution and ROAS: those methods show what happened, not whether marketing caused the result."

Standard retail media reporting tends to overstate performance for two reasons:

1. The first is last-click attribution, which means a sale gets credited to the last ad someone clicked before buying. On sponsored product placements, the shopper is already on the retailer's site, already searching for something in your category. Many of them would have bought anyway. Last-click gives the ad full credit for a sale that was already on its way.
2. The second is identity fragmentation. If a shopper browses on their phone and buys on a laptop, weaker measurement systems count that as two different people. Audiences look bigger than they are, conversions get double-counted, and the ROAS figure ends up more a reflection of how the platform counts than how the campaign performed.

The result is a number that looks healthy in the dashboard but doesn't stand up to a serious conversation about budget.

How does a proper incrementality test work in retail media?

The principle is straightforward: take a group of shoppers, withhold the campaign from a portion of them (the holdout group), run the ads to the rest, and compare what each group bought. The difference is your incremental revenue. In practice, getting a reliable answer in retail media depends on several things.

A real holdout group, not a before-and-after comparison. A proper test needs a defined set of shoppers held out from the campaign at the same time it's running, not a comparison against last quarter's sales. Comparing time periods mixes the campaign effect with seasonality, promotions, weather, and everything else that changes between now and then.

Consider geographic holdouts as a complementary approach. Where user-level holdouts aren't practical, or where you need to measure effects that span in-store and online behaviour across a region, geo holdout testing offers a scalable alternative. This involves splitting matched markets into treatment (ads on) and control (ads off) groups and comparing outcomes across them.

Measurement that follows shoppers into stores. Around 70% of UK retail spending still happens in physical shops. If your test only captures online sales, you're measuring less than a third of the picture and assuming the rest behaved the same way. Resolving the same individual across in-store visits, online browsing, and ad exposure is what makes a holdout test reflect real behaviour rather than the slice of it that happens on a website.

Enough statistical power to trust the result. One of the most common reasons incrementality tests fail is that they're underpowered. If the expected uplift is small, you need large enough test and control groups to separate the real signal from noise. The IAB/IAB Europe guidelines are clear on this point: "a lift estimate that is statistically insignificant, e.g. falls within the noise floor, is not reliable for decision making. A failure to detect a real effect due to small sample size represents inefficient allocation of testing budget."

Enough time for the test to mean something. How long a test needs to run varies by category. Fast-moving everyday groceries can produce a reliable signal in a shorter window than considered purchases like electronics or homewares, where the gap between intent and purchase is naturally longer. The key is to set the test window based on the purchase cycle of the category, not an arbitrary calendar period, and to agree it upfront.

Independent validation of the methodology. A platform measuring its own lift has an obvious conflict of interest. It's worth asking whether your retail media partner's outcomes measurement is independently accredited, because that distinction separates self-reported results from those that have been audited against defined standards.

Where should you start if you've never run an incrementality test?

If incrementality testing is new to you, the easiest first step is to pick one campaign and add a holdout group to it. You don't need to redesign your whole measurement framework on day one. A few things to keep in mind when you do.

Pick a campaign where the answer would change a decision.

A test on a campaign you're going to run regardless is interesting but not actionable. A test on a campaign where the result could justify scaling up, or shutting down, is where the methodology earns its keep.

Agree the holdout group structure before launch.

The size of the holdout, how shoppers are assigned to it, how it's matched to the exposed group, and the statistical thresholds for calling a result all need to be decided in advance. Retrofitting this after the campaign has run gives you a much weaker answer.

Set realistic expectations about what the result will look like.

A well-run incrementality test will almost always produce a smaller number than your reported ROAS. That's not a problem with the test, it's the point of the test. The number is more accurate, even when it's less flattering.

Look at onsite and offsite together.

Offsite placements like display and connected TV often look weak under last-click attribution because they create demand that converts onsite, sometimes days later. Holdout tests that span both channels usually show offsite working harder than the click data suggests. If your retail media partner is running both for you, ask for the incrementality view across the whole campaign, not channel by channel.

What to ask your retail media partner before your next campaign

Before signing off your next campaign report, three questions are worth putting to whoever's running the activity.

1. Can you show me the holdout group structure, including how the test and control groups were built and the statistical thresholds you're using?
2. How does your measurement connect in-store sales back to individual shoppers who saw the ad?
3. Is your outcomes measurement methodology independently accredited?

If the answers come back vague, the ROAS figure you're looking at is an estimate dressed up as a result. A properly run incrementality test won't always make the numbers look bigger. It will make them defensible, and that's the difference between a measurement framework you can build a budget case on and one you can't.

See how incrementality should work

Epsilon's outcomes measurement is independently accredited, and Epsilon Retail Media campaigns are built to measure incremental revenue across online and in-store transactions, not just last-click conversions. If you want to see what proper incrementality testing looks like for your category, talk to our retail media team about running a controlled test on your next campaign.