How to Measure True Incrementality of Paid Campaigns
by Nirjar Sanghavi
|
14 min read
Jul 10, 2026
To measure the true incrementality of paid campaigns, you run a controlled test where a holdout group of your target audience does not see your ads for a defined period, then compare their purchase behavior to the group that did see your ads. The difference in conversion rate between the two groups is your incremental lift: the actual sales your campaign caused rather than sales that would have happened anyway. This matters because every ad platform over-reports its own contribution by claiming credit for conversions that organic search, email, or direct traffic would have driven without any paid spend. Brands that measure incrementality consistently find that 15-40% of platform-reported conversions are not incremental. That is not a rounding error. That is a structural misallocation of budget that compounds every month you do not test it.
DEFINITION: True Incrementality of Paid Campaigns Incrementality measures the additional revenue or conversions that a paid campaign actually caused, compared to what would have happened without it. A campaign has high incrementality if the customers it reached would not have purchased without seeing the ad. A campaign has low incrementality if those customers would have found and purchased from you through organic search, email, or direct traffic regardless. Platform-reported ROAS cannot measure this because every platform claims credit for any conversion within its attribution window, regardless of whether the ad was the actual cause.
Why Does Platform ROAS Overstate Your Campaign's Real Impact?
Every ad platform is measuring the wrong thing, and the incentive structure means they will keep doing it.
When Meta reports a 4x ROAS, it is saying: for every dollar you spent, we observed $4 in revenue from people who clicked or viewed your ad within the attribution window. What it is not saying: how much of that $4 would have happened anyway through other channels.
The structural problem has three components.
Component 1: Attribution windows capture organic behavior. A 7-day click attribution window means that if someone sees your Meta ad on Monday, searches your brand name on Google on Thursday, and purchases directly on Friday, Meta claims that conversion. Google also claims it (via the search click). The customer's purchase decision may have been entirely organic, triggered by a previous purchase experience rather than either ad. Both platforms report the conversion. Neither drove it incrementally.
Component 2: Branded search overlap inflates every platform. Most ecommerce brands running paid social also run branded search campaigns. A customer influenced by a Meta ad will often convert via a branded Google search rather than clicking the Meta ad again. Meta counts a view-through conversion. Google counts the click. The same customer journey appears twice in your reporting. Neither platform tells you this.
Component 3: Post-iOS 14 modeling fills the gaps with estimates. Since iOS 14.5, Meta's pixel cannot directly measure the majority of iOS conversions. It models them using statistical inference. The modeled conversions are presented in the same dashboard as directly measured ones, without distinction. Brands that ran the same Meta spend before and after iOS 14 consistently observed their reported ROAS stay stable while their actual revenue declined. The platform maintained its reported numbers by modeling more conversions, not by driving more sales.
The cumulative effect: platform-reported ROAS is a blend of real incremental impact, organic conversions the platform captured in its attribution window, and statistically modeled conversions that may or may not reflect actual purchases. Untangling these three components requires testing, not attribution model adjustments.
What Is the Most Reliable Way to Measure Campaign Incrementality?
The holdout test is the most reliable method available for measuring true campaign incrementality at the ecommerce brand scale.
How Does a Holdout Test Work?
A holdout test creates a control group: a segment of your target audience that is suppressed from seeing your ads for a defined test period, while the rest of the audience continues to see them normally.
The test structure:
Define your target audience (the population you are currently advertising to)
Randomly assign 10-20% of that audience to a holdout group (suppressed from ads)
Run your campaigns normally to the remaining 80-90%
At the end of the test period, compare the conversion rate of the holdout group to the exposed group
The difference in conversion rate, adjusted for any audience composition differences, is your incremental lift
What the results tell you:
If the holdout group converts at 1.2% and the exposed group converts at 1.8%, your campaign generated 0.6 percentage points of incremental conversion rate, or approximately 33% incremental lift above what would have happened without the ads.
If the holdout group converts at 1.5% and the exposed group converts at 1.6%, your campaign is generating only 0.1 percentage points of incremental lift on a channel claiming much higher attribution. Budget reallocation is warranted.
Statistical requirements:
A holdout test needs enough volume to be statistically significant. As a practical guideline: each group (holdout and exposed) should have at least 1,000 people and run for at least 2 weeks. Shorter tests or smaller populations produce results with too much variance to act on confidently.
For brands spending $30,000 or more per month on a channel, the cost of not running incrementality tests regularly, in terms of budget misallocated to low-incrementality channels, consistently exceeds the complexity of running the tests.
What Other Methods Can You Use to Measure Incrementality?
Holdout tests are the gold standard. Three additional methods provide incrementality signal with different trade-offs.
Geo-Based Testing (Geo Lift Tests)
Instead of splitting an audience, you split geographic regions. One region gets ads; a comparable control region does not. Revenue performance in both regions is compared over the test period.
Pros: Does not require audience suppression technology. Works for channels where holdout audiences are difficult to implement. Cons: Regional differences (weather, competition, seasonality) introduce confounding variables. Requires careful region matching to be reliable.
Best for: TV, out-of-home, or channels where digital holdouts are not available. Also useful for validating holdout test results from a different angle.
Media Mix Modeling (MMM)
A statistical regression model that analyzes the relationship between your total marketing spend by channel and your total revenue over time, controlling for external variables (seasonality, promotions, competitor activity).
Pros: Does not require active test management. Works retrospectively on historical data. Cons: Requires substantial historical data (typically 2+ years of weekly or monthly data) and statistical expertise to build and maintain accurately. Less granular than holdout tests for campaign-level decisions.
Best for: Brands doing $10M or more in revenue with sufficient historical data. Useful as a complement to holdout testing for strategic budget allocation decisions.
Synthetic Control
A more sophisticated version of geo testing that uses statistical methods to construct an artificial control based on weighted combinations of non-test regions or time periods. Used by large brands with access to data science resources.
For most ecommerce brands, holdout testing and geo lift tests provide sufficient incrementality signal without requiring the statistical infrastructure of synthetic control.
How Often Should You Run Incrementality Tests?
The pattern observed consistently among brands that make incrementality testing a standard practice: quarterly tests on their highest-spend channels, with annual tests on lower-spend channels.
Quarterly testing on high-spend channels captures seasonality effects that would distort annual tests. A Meta incrementality test run in Q4 will show very different results than the same test run in Q2, because purchase intent is elevated across all channels during peak periods. Running the test in multiple seasons provides a more accurate picture of the channel's true year-round contribution.
The minimum viable incrementality testing cadence for a multi-channel brand spending $50,000 or more per month on ads:
Meta / Facebook: Every quarter, with holdout group of 10-15%
Google Ads (non-branded): Every six months, with holdout or geo test
Google Branded Search: Once per year, or when spend changes significantly
TikTok / emerging channels: Every six months, to validate whether the channel is earning its budget as it matures
Email / SMS: Annually, using send-time holdouts within your list
Branded search deserves special attention. The pattern observed consistently across ecommerce brands that test branded search incrementality: the true incremental lift is much lower than assumed. Branded searches come from customers who already know you and would find you without the paid result. Most brands significantly over-invest in branded search relative to its actual incremental contribution.
How Do You Interpret Incrementality Results and Act on Them?
Incrementality data produces one of three actionable findings, each with a clear implication.
Finding 1: High incrementality (40%+ incremental lift above baseline) The channel is earning its budget. It is bringing customers who would not have found you otherwise or accelerating purchase decisions that would have been delayed. Appropriate response: maintain or scale spend on this channel.
Finding 2: Medium incrementality (15-40% incremental lift) The channel has real but partial value. Some portion of its reported conversions are incremental; a meaningful portion are cannibalized from organic or other channels. Appropriate response: hold spend steady, investigate whether creative or audience adjustments can improve the incremental ratio, retest in 90 days.
Finding 3: Low incrementality (under 15% incremental lift) The channel is primarily capturing organic demand rather than creating new demand. Cutting or pausing this channel would cause minimal revenue loss while recovering significant budget. Appropriate response: reduce spend by 30-50% and monitor blended MER over 30 days to confirm the revenue impact is smaller than the spend reduction.
BI Reportingthat connects your ad spend to your actual revenue outcomes, including the holdout group comparison data, makes this analysis accessible without requiring custom data pipelines. A platform that unifies your ad channel data and your Shopify or Amazon order data in one place gives you the baseline for interpreting incrementality results in context rather than in isolation.
What Is Blended MER and How Does It Complement Incrementality Testing?
Blended MER (Marketing Efficiency Ratio) is total revenue divided by total marketing spend across all channels, calculated at the business level rather than the platform level.
MER is not a substitute for incrementality testing. It is a complement.
Here is how the two work together:
Run incrementality tests to identify which channels have low incremental lift.
Reduce spend on low-incrementality channels.
Monitor blended MER over the following 30-45 days.
If blended MER holds steady or improves after reducing spend on a low-incrementality channel, the test result was correct: the channel was capturing organic demand that continued without the paid support.
If blended MER declines materially after reducing spend, the channel had more incremental impact than the test suggested, or the reduction exposed a gap in another channel. In either case, MER gives you a business-level feedback signal that confirms or challenges the incrementality finding.
AI Agentsthat monitor blended MER and flag significant changes automatically give you this feedback signal in real time rather than requiring you to check manually after each budget change. The combination of planned incrementality testing and continuous MER monitoring is the most reliable system for ongoing budget optimization.
The Incrementality Ladder
THE INCREMENTALITY LADDER: A four-rung framework for progressively improving the accuracy of campaign incrementality measurement as a brand scales its ad spend and testing sophistication.
Here is how it works. Not every brand is ready for holdout testing on day one, and not every channel warrants the same testing rigor. The Incrementality Ladder gives operators a sequenced approach that matches measurement sophistication to spend level:
Rung 1: MER baseline (any spend level). Before running any incrementality tests, establish a reliable blended MER baseline across all channels. This is your north-star number and the reference point against which all future test results are interpreted.
Rung 2: Geo lift tests ($20K-50K/month). At this spend level, simple geo-based tests on your top one or two channels provide incrementality signal without requiring audience suppression technology. Run one test per quarter on your highest-spend channel.
Rung 3: Holdout tests ($50K-150K/month). At this spend level, the cost of running holdout tests (typically 10-15% of audience suppressed for 2-4 weeks) is justified by the precision of the results. Run quarterly on Meta and Google non-branded. Run annually on branded search.
Rung 4: Continuous testing program ($150K+/month). At this spend level, incrementality testing should be an ongoing operational practice rather than a periodic initiative. Multiple tests run simultaneously, results feed directly into weekly budget allocation decisions, and MER is monitored daily.
The Incrementality Ladder, developed from patterns observed consistently across ecommerce brands at different stages of ad spend sophistication, establishes that the right measurement approach for your business is not the most rigorous approach available. It is the most rigorous approach your current spend level and infrastructure can support reliably.
Conclusion and CTA
Measuring the true incrementality of paid campaigns is not a theoretical exercise. It is the mechanism by which you find out whether your ad spend is creating demand or just taking credit for demand that already existed.
The brands that do this consistently reallocate 20-30% of their ad budgets within the first year of regular incrementality testing. That is not because they were wasting money carelessly. It is because platform attribution made low-incrementality channels look better than they were, and nobody tested the assumption.
The Incrementality Ladder gives you the right starting point for your current spend level. Blended MER gives you the feedback mechanism to validate what the tests find. Together, they are the system that turns ad spend from a cost into a compounding advantage.
Q1: What is incrementality in paid advertising and why does it matter?
Incrementality measures the additional revenue a paid campaign actually caused, versus what would have happened without it. It matters because every ad platform claims credit for conversions within its attribution window, including conversions driven by organic search, email, or direct traffic. Brands that measure incrementality consistently find that 15-40% of platform-reported conversions would have happened without the paid campaign, meaning that budget is not generating new revenue.
Q2: What is a holdout test and how do you run one for paid campaigns?
A holdout test randomly suppresses ads from 10-20% of your target audience for a defined period, typically 2-4 weeks, while the remaining audience sees ads normally. At the end of the test, you compare the conversion rate of the holdout group to the exposed group. The difference, expressed as a percentage, is your incremental lift. Each group needs at least 1,000 people to produce statistically reliable results.
Q3: How do you know if your paid campaigns have high or low incrementality?
Run a holdout test and compare the conversion rate of the holdout group to the exposed group. High incrementality is generally defined as 40% or more lift above baseline: the exposed group converts at a meaningfully higher rate than the holdout. Low incrementality (under 15% lift) means the channel is primarily capturing organic demand. A 30-50% spend reduction on a low-incrementality channel typically causes a smaller revenue decline than the spend reduction, confirming the test result.
Q4: Can blended MER replace incrementality testing?
No. Blended MER (total revenue divided by total marketing spend) is a business-level efficiency metric that tells you overall marketing performance but cannot identify which specific channels are incremental. Incrementality tests identify which channels are creating demand versus capturing organic demand. The two are complements: use incrementality tests to inform budget allocation, use blended MER to validate the results of reallocation decisions over the 30-45 days that follow.
Q5: How do you measure incrementality for branded search campaigns?
Branded search incrementality is measured using a geo lift test: suppress branded search ads in one geographic region while running them normally in a comparable region, then compare branded search revenue (plus any organic branded traffic increase) between the two regions. The pattern observed consistently across brands that test this: branded search has lower incremental lift than most brands assume, because customers searching your brand name would find your organic listing without the paid result.
Q6: What data infrastructure do you need to run incrementality tests?
You need three things: a unified analytics platform that connects your ad spend data to your actual order data (so you can measure revenue outcomes, not just ad platform conversions), the ability to segment and suppress audiences within your ad platforms, and a baseline period of historical MER data for comparison. Trivas.ai connects ad channel data with Shopify, Amazon, and other order sources in one platform, which gives you the unified revenue view that makes incrementality results interpretable in a business context rather than as isolated ad metrics.
Q7: How much ad spend do you need before incrementality testing is worth doing?
Geo lift tests are viable at $20,000 or more per month on a channel. Holdout tests with statistically reliable results require enough audience volume to put 1,000 or more people in each group for 2-4 weeks, which typically means $50,000 or more per month on the channel being tested. Below those thresholds, MER monitoring and qualitative creative analysis provide more actionable signal than underpowered incrementality tests.
Q8: How do you act on incrementality test results without disrupting your business?
When a test reveals low incrementality, reduce spend gradually rather than pausing the channel entirely. A 30-50% reduction on a low-incrementality channel is a diagnostic step, not a final decision. Monitor blended MER for 30-45 days after the reduction. If MER holds steady or improves, the test result was correct. If MER declines more than the spend reduction would predict, the channel had more incremental impact than the test captured and the reduction should be partially reversed. Trivas.ai's continuous MER monitoring provides the real-time feedback signal that makes this validation process systematic rather than dependent on manual weekly checks.
Visionary leader with 20+ years of deep expertise in eCommerce analytics and business intelligence at companies like Samsung, Groupon, eBay, PayPal, and Chase. Nirjar founded Trivas with the mission to democratize data-driven decision making for online merchants.
Continue Reading
explore more insights
The Practical Guide to Automate Your Ecommerce Reporting
3 min read
Technology and Platform Considerations
3 min read
Daasity vs Polar Analytics: 8 Differences Founders Should Know
.d53b12e5.png&w=3840&q=75 "Trivas.ai")
by Nirjar Sanghavi
