We’ve written about the value of incrementality testing, how to run  incremental lift testing and even which methodologies are the most efficient and cost-effective. Wondering how running an incremental lift test can help your brand’s reach its goals? Below, go behind the screen to learn a real-world example of how we’ve tested for incremental lift in the past.

Quick review: what is incrementality testing?

To put it briefly, incrementality testing is the best measure for ad success. Why? Because incremental lift testing uses causal effect analysis to isolate the value of their paid mobile advertising campaigns, independent of organic growth efforts. This is valuable to mobile marketers for a few reasons. First, it lets marketers prove the value of their paid marketing efforts so that they can win more budget. Second, it enables marketers to understand whether their paid efforts are cannibalizing organic growth they may have achieved anyways. Finally, with incrementality testing, app advertisers can spend their ad dollars more efficiently by understanding what’s driving ROI in their campaigns.

Get started with incrementality testing

When looking to get started with incrementality testing, make sure the DSP you’re working with can test for incremental lift before you’ve launched a campaign with them. Keep in mind, an incrementality test should be run with at least one to two months worth of campaign data. To collect the most unbiased data, DSPs will often launch a campaign and keep the parameters relatively unchanging. In other words, they’ll touch a campaign as little as possible in order to get a better understanding of whether their initial optimizations showed positive uplift results.

How we use incrementality testing

Our goal is to demystify the incrementality testing process so that more marketers leverage this valuable tool. Here’s a real example of how we’ve used incremental lift testing for a partner in the past.

An example use case of incremental lift testing

A mobile game app is looking to prove the value of their paid retargeting campaigns with a DSP. Before setting their campaign live, they confirm with the DSP that they can run  incremental lift testing. Within the first two months of the campaign, the DSP achieves the mobile game app’s goals. However, the mobile game app would like to understand if any of this performance is related to their organic growth. They would also like to understand if there are optimizations they can make to enhance the performance of the campaign. After the campaign has been live for two months, the DSP runs the test.

What the incrementality test is evaluating 

The mobile game asks the DSP to run incremental lift testing to evaluate the effect their paid retargeting campaigns have had on their return on investment (ROI).

Reporting methods

At YouAppi, we provide different types of reporting on incremental lift to advertisers based on their goals.

The first type of report we use is a week-by-week or “cohorted” report. This reporting method uses an “anchor” — a period of a week — to monitor reactivated users on a weekly basis. So, all users re-activated within a specific anchor — e.g. within a specific week — are tracked for up to a month. Data is collected on whether they made a transaction, churned, etc. on Day 1, Day 3, Day 7, Day 14 and Day 30. So, if a user re-opened and re-engaged with a game app on January 1st, but didn’t make a purchase until January 25th, then the purchase is still associated with the January 1st anchor period.

The second type of report is a monthly report. In contrast to the weekly or cohorted report, the monthly report captures all information about reactivated users and their behavior within a month period versus a week-by-week split. Furthermore, in the monthly report, uplift data can be organized by campaign and segment.  This is in contrast to the cohort report which only contains uplift information that’s organized week by week.

At YouAppi, we generally run both reports to get a comprehensive understanding of a campaign’s performance.

The process

To set up an incrementality test, the DSP takes a random selection of each audience segment and splits them into two groups: 10% in the control group and 90% in the test group. The test group receives a treatment while the control group does not. The difference in the revenue generated by both populations is then measured for incremental lift.

At YouAppi, to limit the “noise” in our results, we test for incrementality at the bid request level. This involves placing ghost bids — or, invisible bids — on users in both the control and test groups to create an apples-to-apples comparison between users who ‘would have been exposed’ and ‘would not have been exposed’ to an ad.

For more information on why ghost bidding is the most effective method for incrementality testing, read our blog on how to run an incrementality test here

Keep in mind that depending on the size of your app’s audience, a DSP might recommend running incrementality tests less frequently.  This is so they don't continue taking 10% chunks of your relevant audience for the holdout group.

The results

incrementality testing

According to the results, the DSP drove a 17-29% uplift in ROI from Day 1 to Day 7 after re-engaging a player. A 17.5% uplift means the DSP covered 100% of the mobile game’s campaign costs and an additional 17.5% in revenue. So, the DSP is not only  hitting the mobile game’s goals, it’s also driving an over 17% lift in revenue. This revenue is being driven by paid retargeting campaigns specifically — since the incrementality test has isolated and measured paid performance separately from organic performance. From the mobile game’s viewpoint, these positive results show the DSP is driving value. It also shows that if they don’t invest in retargeting, they’ll lose out on additional revenue.

Campaign recommendations

Along with proving the value of the paid retargeting campaign, the incrementality test also provided the DSP with information about how to better spend the game app’s ad budget.

Using a monthly report, the DSP tested uplift on specific segments of the campaign. It found that users with a lifetime value of $25 who had been reactivated within the last 30 days showed a 40% uplift in revenue compared to users that had not made a transaction. The DSP recommends users with a LTV of $25 are segmented and served specific ad creatives to encourage further purchases. In this way, the incrementality test is used to make recommendations on how the game app can spend their ad dollars more effectively.

Takeaways

Incrementality testing uses causal effect analysis to isolate the value of paid mobile advertising campaigns. Furthermore, it also evaluates how to optimize campaign spend for enhanced performance.

An example use case of incremental lift testing:

  • A mobile game app wants to prove the value of their paid retargeting campaigns with a DSP. To do this they ask the DSP to run  incremental lift test. The test evaluates the effect their paid retargeting campaigns have had on their ROI.
  • The results. The DSP drove a 17-29% uplift in ROI with retargeting from Day 1 to Day 7 after re-engaging a player. A 17.5% uplift means the DSP covered 100% of the game’s campaign costs and an additional 17.5% in revenue. Additionally, with a monthly report, the DSP determined that users with a LTV of $25 or more reactivated within the last 30 days showed a 40% uplift in revenue compared to users that had not made a transaction.

Use incrementality to prove the value of your paid retargeting campaigns

From campaign launch to incremental lift testing, our experts can help you understand what’s driving performance in 2023. Schedule a meeting with us to get started.