Many businesses look at the ROAS on their ad dashboard and conclude their ads are worth it. But the more important question is: if you didn't run this ad at all, would this customer have bought anyway? Incrementality testing is the only scientific way to answer that.
The problem with dashboard ROAS
The conversions ad platforms report often include customers who would have bought anyway without seeing an ad, such as existing customers searching your brand name on Google, or people who already intended to buy and happened to see a retargeting ad first. The system credits all of these to the ads, even though the sales might have happened regardless.
What is a holdout test, and how do you run one?
The most popular method is a holdout test or geo test: split your audience or geographic areas into two groups. One group sees your ads as normal, while the other is held back from seeing any ads (the holdout group). Then compare sales between the two groups over the same period. The difference is the incremental lift, the sales genuinely created by your ads.
PSA (Public Service Ad) tests on Facebook
Facebook's Conversion Lift Study uses the same principle as a holdout test, showing some users a public service announcement (PSA) instead of the brand's real ad, then measuring the difference in conversions between the two groups. It suits businesses with large ad budgets that want formal proof of Facebook Ads performance.
How can small businesses run a simple incrementality test?
SMEs without the budget for big platforms' lift study tools can run a simple geo-based test themselves, for example by pausing ads in certain provinces or areas for 2–4 weeks and comparing sales with areas where ads keep running. No special tools are needed, just the sales-by-area data you already have.
- An incrementality test tells you whether ads create additional sales, not just whether they're associated with sales
- Holdout and geo tests work for businesses of any size, and don't require a big budget
- Repeat the test every 3–6 months, because market behaviour keeps changing
- The results may reveal that some channels that look profitable actually have less impact on sales than you thought
Summary
Knowing your ROAS alone isn't enough for long-term budget decisions. The Markettung team designs incrementality tests that fit each client's business size, so you can be sure every baht invested in ads creates measurable results.