GUIDE
The mobile marketer’s guide to incrementality analysis
Attribution remains the critical basis of marketing measurement, but it doesn't tell the full story. As marketers face growing pressure to improve efficiency and prove return on investment (ROI) despite decreased access to device-IDs, understanding whether a campaign actually caused an outcome has become just as important as knowing where a conversion was attributed.
This is where incrementality analysis comes in. By measuring the causal impact of marketing activity, incrementality helps marketers determine whether campaigns generated additional value, improved user acquisition (UA) outcomes, cannibalized existing demand, or had little measurable effect at all.
In this guide, we explain what incrementality is, how it works, and how mobile marketers can use it alongside attribution to make more confident campaign optimization and investment decisions.
Incrementality definition
What is incrementality in marketing?
Incrementality is a measurement methodology used to determine the causal impact of a marketing activity on a specific business outcome or key performance indicator (KPI) such as installs, sessions, in-app events, conversion rates, level completions, and countless more.
)
Rather than simply identifying where a conversion was attributed, incrementality helps marketers understand whether a campaign actually generated additional value that would not have occurred otherwise. In other words, it answers a critical question:
What would have happened if this marketing activity had never taken place?
By comparing observed results against an estimated baseline, marketers can quantify whether a campaign:
- Generated incremental lift
- Cannibalized existing organic demand
- Had no statistically significant impact
This makes incrementality an important complement to attribution. While attribution tells marketers where conversions came from, incrementality reveals whether marketing activity actually caused those conversions.
When applied correctly, incrementality analysis enables marketers to make more confident decisions about budget allocation, channel investment, campaign optimization, and long-term growth strategy.
Understanding incrementality outcomes
The results of an incrementality analysis typically fall into one of three categories:
)
Your incrementality test results will tell you the exact, direct impact of an individual marketing effort on a given KPI. That might look a little something like this:
Incrementality can also reveal more nuanced insights across platforms, markets, channels, and campaign types.
Measuring incrementality
How to measure incrementality
Incrementality analysis can be done in a few different ways. At its core, incrementality works by leveraging causal data science models and control groups (more on this below), which tells us what would have happened in the absence of a marketing action. In other words, no more guessing, “What would happen if I doubled my budget with network A, or stopped my search campaigns entirely?”
Traditionally, the approach was to segment a target audience into a control group and a test group. The test group was exposed to the marketing campaign, and the results were compared against the control group. In essence, A/B testing. A campaign either runs or it doesn't. Incrementality measurement bridges this gap by estimating the counterfactual outcome: the result that would likely have occurred without the campaign.
)
While this approach remains highly effective in many situations, it can be difficult to execute at scale. Challenges often include:
- Maintaining clean separation between test and control audiences
- Preventing exposure to overlapping campaigns or external influences
- Building statistically reliable control groups
- Pausing or restricting marketing activity during the testing period
For organizations operating across multiple channels, geographies, and campaigns simultaneously, these requirements can introduce operational complexity and limit testing velocity.
InSight: The Adjust approach to incrementality
Adjust strives to offer data and insights that are as accurate as they are reliable, in a privacy-centric manner. So, we tweaked this traditional take and molded it into something truly future-proof for the modern marketer. Adjust’s approach to incrementality analysis, through our InSight solution, has two key differences. InSight:
- Uses aggregated historical attribution and ad spend data, combined with data observed from significant external factors, to create a synthetic control group.
- Leverages a machine learning model to make the comparison between the test and control data groups with a confidence interval of 95%.
For each analysis, the synthetic control group will closely correlate with the app in question in both app metrics and user behavior. This also ensures that no marketing campaign needs to be paused to find true incremental lift.
InSight
This approach offers several advantages:
- No need to pause campaigns or withhold marketing activity from audience segments
- Scalable testing across channels, campaigns, markets, and platforms
- Privacy-conscious measurement built on aggregated data
- Faster access to actionable insights for optimization and budget allocation
For each analysis, the synthetic control group will closely correlate with the app in question in both app metrics and user behavior. This also ensures that no marketing campaign needs to be paused to find true incremental lift.
)
By estimating incremental impact alongside attribution data, marketers gain a clearer understanding of marketing effectiveness. Instead of evaluating performance solely through reported conversions, they can assess whether a campaign influenced outcomes that would not otherwise have occurred and make decisions with greater confidence.
Incrementality analysis use cases
The value of incrementality analysis extends beyond campaign measurement. Depending on the business objective, it can be used to evaluate everything from channel performance to seasonal marketing initiatives. There are many ways and a few specific ways Adjust’s method of incrementality measurement can be leveraged. Here are some of the most common ones.
- Seasonality testing: Validate how effective seasonality is to, for example, increase budget around big holidays.
- Advertising channel discovery: Validate the efficacy of new advertising channels when it comes to certain events.
- Platform-based optimization: Understand if the channel in question yields better results for iOS or Android.
- Ad fatigue detection: Verify if a campaign has reached ad fatigue.
)
Examples
Understanding the ins and outs of Adjust’s InSight for incrementality analysis
Here's what a typical incrementality analysis looks like in practice.
- First, a marketer identifies which marketing action, campaign, country, and timeframe the analysis should be for. Then, they identify if the incrementality test will be analyzing performance based on installs or events.
Example: I want to validate the performance of network A in the U.S. when it comes to converting users to make a first purchase during the week before “first day of school”.
2. Then, the testing period is chosen. There must be enough historical data to calibrate the machine learning models accurately, which means the testing period requires:
- 12 weeks of ad spend data
- At least 100 installs or events per day on average (depending on the target variable you choose to test).
3. Next, the marketer works to eliminate as many variables as possible. Any ongoing campaigns that were part of the analysis period are fine, but it’s important to avoid introducing any new channels, campaign formats, or actions that were not recorded in that 12-week period before analysis.
This means, for the seven days before and after the incrementality test, do NOT:
- Start any new campaigns or marketing actions.
- Changing the budget of ongoing marketing campaigns.
Then, incrementality analysis can only occur once seven days have passed from the initial trigger of the marketing action. This is so that the model has enough data to precisely predict what the outcome would have been without the marketing action.
In the case of Adjust’s InSight incrementality model, patterns in installs or events are recognized with input data like:
- App store ranking
- Ad spend
- Installs
- Monthly active users
- Industry trends and significant external factors
- Daily sessions
These data points will be available in the next phase of InSight. Stay tuned!
The result? A clear and straightforward indication that your marketing action either caused:
- Incremental lift (more installs or events were conclusively due to a particular marketing action).
- Organic cannibalization (the increase of installs or events was only due to stealing it from organics because they were going to happen anyway) or statistically significant loss as a result.
- Or, no incremental value (no significant movement, positive or negative, detected).
)
Incrementality in action
Real-world results with InSight
Organizations use incrementality analysis in different ways depending on their measurement goals. Whether validating channel performance, improving budget efficiency, or running controlled experiments, the objective remains the same: gaining a clearer understanding of marketing effectiveness.
Here are a few examples of how Adjust customers have applied incrementality methodologies in practice.
Sleep Cycle
Sleep Cycle, a leading sleep tracker app, was one of the first apps to harness InSight to refine and scale its marketing strategies without cannibalizing its strong organic traffic. Despite challenges posed by Apple's SKAdNetwork (SKAN) and the need for precise insights for product innovation and hyperlocalization, Adjust provided Sleep Cycle with the tools to measure the true impact of marketing activities.
Through Adjust's innovative approach, including the creation of synthetic control groups to navigate the post-device ID era, Sleep Cycle gained critical insights. This enabled them to experiment with new channels, optimize campaigns, and make data-driven decisions to avoid organic cannibalization and maximize investment in effective marketing channels.
Linnéa Gosh
Digital Marketing Analyst, Sleep Cycle
Consequently, Sleep Cycle has strengthened its market position, optimized its marketing spend, and maintained privacy-conscious practices, ensuring sustainable growth and enhanced marketing efficiency.
Fabulous
As market conditions and privacy requirements continued to evolve, Fabulous partnered with Adjust to strengthen its measurement strategy through marketing mix modeling (MMM) and incrementality analysis. By establishing a clearer baseline for performance, the team was able to evaluate marketing effectiveness more accurately and focus on spending efficiently rather than simply increasing budgets.
)
Filippo de Rose
CMO, Fabulous
Careem
Careem used Adjust's Audiences to design an incrementality experiment that measured the impact of TikTok campaigns on existing app users. By creating treatment and control groups and comparing outcomes across both audiences, the team was able to assess the incremental effect of advertising on product discovery and first-time grocery orders.
){kind=logo}
Hassaan Ali
Senior Manager Performance Marketing, Careem
Adjust clients Pret-a-Manger and iPocket Games also saw success implementing incrementality testing with InSight.
Jay Christian
Performance Marketing Manager, Pret-a-Manger
Sakura Hong
Marketing Director, Hong Kong iPocket Technology Ltd
Ready, set, benchmark
Benchmarks from additional clients using InSight:
- 50% of channels tested showed cannibalization
- Most cannibalization came from gaming and e-commerce verticals
Benefits
The benefits of using incrementality analysis
As the industry moves away from granular user-level data towards a more aggregated methodology of measuring marketing efforts, marketers’ ability to surface details in real-time and understand the true impact of their efforts naturally is further complicated and hindered.
Yet, more is being asked of them. More precision, more agility, more efficiency, and more results. Adding incrementality analysis to a marketer’s tool belt can be game-changing in this complex landscape, as it provides a number of impactful benefits.
Budget agility
Confirming what’s lifting performance and what’s not with a confidence interval of 95% makes it easy to optimize budget spending and changes as frequently and swiftly as needed for the highest performance.
Immunity against industry changes
Since incrementality is privacy-centric and data-aggregation-based, the incremental impact of campaigns can be measured as national and global privacy changes continue to roll out.
Resources and time reduction
Without the need for a dedicated data science team to comb through data on a daily basis, those resources can be freed up to focus on other matters. Better yet? Not having to invest the money in an in-house incrementality analysis or 3rd-party solution while still getting valuable insights about campaign performance right from the comfort of the Adjust dashboard.
Confidence and support to innovate
Having a precise way to validate new channels, ad formats, countries, seasonality, and other alternative marketing tactics allows marketers to confidently leave their comfort zone to expand their reach and audience and diversify marketing strategies.
Partner accountability
With more visibility into organic cannibalization, marketers can have honest conversations with ad networks when it comes to making the necessary optimizations to achieve success with them.
)
Best practices
Best practices for optimal incrementality testing
The accuracy and usefulness of an incrementality analysis depend on more than the model itself. Following a few key best practices can help improve the reliability of your results and increase confidence in the insights they generate.
Start with a clear hypothesis.
Be as specific as possible about the outcome you were hoping to get. For example, if you’ve decided to double your budget with network A, your hypothesis might be a 10% increase in Android installs in China.
Test campaigns with identified impact.
Because you’ll have historical data, make sure to first identify a causal relationship between your campaign and KPI. Then, you’ll be able to measure this impact with incrementality analysis. This means that if your campaign didn’t move the needle, then it’s not worth testing for incremental value.
Know how to preserve data reliability.
If you’re already an Adjust client, we’ve got you covered! Any activity for which we already have ad spend data can be fed into our models. If you’re running marketing campaigns that don’t automatically have ad spend measured by us (either offline, on your own channels, via affiliates, etc.), then you must make sure that Adjust is getting the ad spend information. Or, avoiding running those particular campaigns during the pre- and post-analysis periods.
Focus on business outcomes, not just acquisition metrics.
Many marketers begin with installs because they are easy to measure, but incrementality analysis can be applied to outcomes throughout the customer lifecycle. When possible, align tests with metrics that reflect meaningful business value, such as purchases, subscriptions, retention, or engagement.
Test one variable at a time.
Incrementality analysis is most effective when the marketing activity being evaluated is clearly defined. Simultaneously changing budgets, creatives, targeting strategies, and channels can make it more difficult to understand what actually influenced the outcome.
Treat incrementality as an ongoing practice.
A single test provides a snapshot in time. Consumer behavior, competitive activity, and marketing performance can all change over time, making it important to revisit assumptions and validate findings regularly.
Use incrementality alongside attribution.
Incrementality and attribution answer different questions. Attribution helps explain where conversions were credited, while incrementality helps determine whether marketing activity influenced outcomes. Using both methodologies together can provide a more complete picture of performance.
Next steps
A more comprehensive view of marketing performance
One of the biggest challenges in marketing measurement is that observed performance and incremental impact are not always the same thing. A campaign can receive attribution without creating additional demand, while a channel can appear effective despite capturing conversions that would have happened anyway. Understanding this distinction is critical for marketers looking to make informed investment decisions.
Incrementality analysis helps address this challenge by estimating what would have happened in the absence of a marketing activity. By establishing a credible baseline and evaluating performance against it, marketers gain context that attribution alone cannot provide. This enables a more complete understanding of marketing effectiveness and helps organizations assess the true impact of campaigns, channels, and budget decisions.
As measurement continues to evolve, marketers who incorporate incrementality into their measurement framework will be better positioned to evaluate performance with greater confidence and make decisions based on a deeper understanding of cause and effect.
Ready to see how Adjust InSight can help, or how we can grow your app business in general? Request a demo to learn more.
Want to get the latest from Adjust?
Keep reading
