# Lies, ROAS lies and Statistics

There is a simple idea at the core of most mobile marketing campaigns these days – if you spend \$x on some marketing activity and received \$y in return you want y to be grater than x. This is often referred to as ROAS or campaign ROI. We have trained mobile marketers to break down their activities to small units: ad groups, ad sets, ad creatives, audiences, … and find the ones that show ROAS. Doubling down on the positive ROAS units while shutting down the negetive ROAS units is the leading campaign optimization strategy today.

Here is the problem – it only works under certain conditions.

There is a famous saying by Mark Twain – “There are lies, damned lies and Statistics”. It comes to warn people about using statistics in a wrong way. One such way is using statistics when small numbers are involved. Another way in which statistics are deceiving is called Multiplicity or Multiple comparisons. Let’s see how those come into play when calculating returns.

### Beware of the small numbers

Most companies base their ROAS calculations only on revenues from In-App Purchases. This is a result of 2 things:

• Up until recently, ad based monetization and ad spend were mutually exclusive
• Until SOOMLA TRACEBACK there was no way to attribute ad monetization

The problem with In-App Purchases revenue is that it’s highly concentrated. Studies have shown that purchases are less than 2% of the users and among those 2%, the top 10% generate half of the revenue. Let’s say that you spent \$5,000 to acquire 1,000 users and you are trying to figure out the return. Most likely you have 20 purchases but there are 2 whale users who generated \$1,500 each (this is aligned with the studies – yes). Now, suppose you had 2 ad-groups in that campaign and you are trying to figure out which one was better. Here are the options:

• Group A had both whales