There are countless benefits to incrementality testing. From proving the value of your retargeting spend to helping you understand how to best spend your next ad dollar, incremental lift testing is a key component of any robust mobile advertising campaign. But, did you know that incrementality testing can also help you segment your audience for supercharged performance? Below, learn how we use incremental lift testing to identify high-performing segments and allocate ad spend more effectively.
For a review of incrementality testing, read our blogs on what incrementality is, how to execute an incremental lift test and how incrementality can help you optimize your next ad dollar.
Incremental lift testing example
With incrementality testing you can compare the relative uplift and value of different targeting strategies. This, in turn, can help your team prioritize certain strategies and spend your marketing budget more efficiently.
A game app wants to target high-performing users
To illustrate how incrementality testing can help apps better segment their campaigns, here’s an example:
A game app is looking to better understand what segments of users to target to drive performance in their retargeting campaigns. To do this, they have their DSP randomly select 10% of their audience to place in a control group. Then, they run incrementality uplift on the treatment and control groups of their campaigns. They also run uplift tests on all the specific segments within each campaign. These tests are executed at least two months after the campaign was launched.
A note on control groups
It’s crucial to make sure the data in your control group is cleaned as much as possible prior to running your uplift test. This means fixing or removing incorrect, corrupted, incorrectly formatted, duplicate, or incomplete data within the dataset. If data is incorrect, outcomes and algorithms are unreliable, even though they may look correct. There are a few ways to clean data. For example, you could work with one control group with all vendors or clean it up post-analysis.
Incremental lift at the campaign level
While on an overall app level, the uplift test shows a positive result — indicating the retargeting campaign is driving value — on a campaign level, the incremental lift test shows one campaign had a negative uplift on revenue.
The campaign targeting non-paying users on Android that have engaged with the app within 7 days had a negative lift. This means that retargeting these users did not drive a lift in revenue. That being said, this result was based on a smaller amount of data than the other campaigns. The app’s DSP digs into the results of their incrementality tests on a segment-by-segment basis to better understand the nuances of these results.
Incremental lift at the segment level
Flushing out the campaign segment by segment showed that some segments were contributing more to the overall uplift in revenue than others. For example, the segment retargeting iOS non-paying users that had engaged with the app within 90 days drove an over 5000% uplift in revenue. That being said, this segment had also collected a smaller amount of revenue data compared to other segments.
The segment with the most data — Android paying users that had last logged in within 7 days and engaged with the app within 30 days — also showed a high uplift of 110%. This reflected how retargeting users in this segment drove a more than 100% lift in revenue for the game app.
On the other hand, Android paying users who had logged in within 7 days but engaged with the app within a slightly longer window of 90 days showed a negative uplift of 50%. This means that retargeting these users — that might have logged into the app within 7 days but before that last engaged within a longer 90 day window — did not result in a positive lift in revenue.
What the game app learned
Deep diving into the incrementality results not just on a campaign level but a segment level gave the app a more nuanced understanding of the users most influenced by their retargeting efforts. For example, on a campaign level, paying users on Android who had last logged in within 7 days showed a positive uplift. But, when the app’s DSP dove deeper into these results on a segment level, they found nuances to this performance. Namely, paying users on Android that had logged in within 7 days but been active within 30 days had a higher uplift when retargeted. This was in contrast to paying users on Android that had logged in within 7 days and been active within 90 days. This group showed a negative uplift.
The game app learned from these results that after a 30 day recency the potential uplift of retargeting a player to come back and re-open their app decreases. They also identified segments with high potential that they could give more spend to. This included non-paying users on iOS with a 90 day recency, active users on iOS who had logged in within 7 days with a 30 day recency and non-paying users on iOS with a 30 day recency.
Takeaways
The biggest takeaway from this use case? The importance of analyzing the results of your incrementality testing both on a campaign and segment level. In addition to this, here are some best practices for segmenting your audience based on incrementality testing.
- Collect enough data. As outlined above, not all incremental lift results are the same. You might have a highly positive or negative lift that’s not necessarily backed by a large amount of data. Don’t jump to conclusions as you analyze your results. Keep in mind they could change as you collect more information.
- Remember that lift doesn’t always equal performance. Depending on your app’s KPIs, a positive uplift might not equate to high performance. For example, you might see a positive revenue uplift in your incrementality testing but still be hitting below your LTV or churn goals. Make sure you’re connecting your incrementality testing to your down funnel goals. Furthermore, uplift should be taken as another way to view your campaign from a different angle. That being said, it’s crucial to analyze your attributed data in parallel to your uplift results.
- Keep in mind cost. Unless you’re passing back cost data to your DSP, your incrementality results might not reflect the ROI results of your campaigns. For example, you could see a high uplift in conversions while your cost per conversion is also very high. Keep in mind cost and make sure that before you allocate ad spend to a certain segment you consider how much the users in that segment cost to reactivate.
Retarget your audience for supercharged growth
From acquisition to retargeting and incrementality testing, YouAppi’s team of experts can help you drive revenue for long-term performance. Schedule a meeting with us to get started.