Why vanity metrics really need to die
Liam Shaw, Wrappr's co-CEO, argues why advertisers should make it their new financial year resolution to measure metrics in a more meaningful way.
While it may not be as resolution-filled as January, the end of the financial year is a time of to reflect on what is and isn’t working. Let’s begin with metrics.
Marketers have plenty of metrics to get lost in; impressions, reach, frequency, and media mentions all sound good on the surface. However, none of these ‘vanity metrics’ matter unless they are contributing to the ultimate purpose of the organisation.
Picture: Karolina Grabowska
When making decisions with wide and sweeping impact, such as a marketing budget or media plan, it’s important for businesses to be looking at the metrics that irrefutably relate directly to objectives rather than those that look good at first glance.
Don’t confuse correlation with causation
It’s important to ask how the marketing mix impacts sales. However, this is a question that’s easy to answer incorrectly. Many of the metrics that advertisers use imply causality when they really only measure correlation.
ROAS (return on ad spend), for example, is often misleading because if someone buys something soon after an ad has reached them most platforms track this as a ‘win’. Therefore, ROAS can take credit for another channel’s earlier work.
Audience targeting algorithms that are geared to maximise ROAS can also prioritise people who are close to purchasing or are existing customers that frequently purchase, generating easy sales rather than growth. ROAS prioritises efficiency over effectiveness, and it can lead to businesses staying small.
Sales growth is another metric that, when it correlates with marketing activity, is often claimed by a marketer eager to demonstrate their value, but upon further scrutiny such claims become difficult to support.
Conversely, when sales don’t grow you’ll be hard pressed to find a marketer who will claim that this is their win, but what if their efforts actually stopped the business declining?
The only way to truly determine marketing’s impact on sales is through single-source data, control groups and exposed groups (more on this later!).
Context, please
Metrics need to be measured with other metrics for context. For example, if you give people KPIs to work towards and put too much emphasis on them, then people can end up optimising for that KPI past the point of benefits.
One example is how ROI can be taken too far. When marketers target advertising only at those customers who are likely to purchase in the short-term and cut spending from advertising that will bring in new customers to the category the result will be that their ROI will go up. Hooray, goal achieved! But it will have been at the sacrifice of bringing in less sales overall.
Therefore, the KPI of improving ROI needs to be provided with the context of other KPIs that will ensure that long-term sales won’t be sacrificed such as a brand tracking funnel that shows how awareness, consideration and intent to purchase are changing.
On top of this are critical profitability metrics like CAC (cost of customer acquisition), CLV (customer lifetime value), and CAC payback (which is the amount of time it takes to break even on a new customer). In this economic climate, demonstrating profitability will be how marketers win larger budgets.
It’s time to be a scientist
Purchasing decisions rarely happen due to one advertising exposure, and it doesn’t make sense for one provider to take the credit for a purchase just because the customer clicked their ad.
This is where marketing mix models (or MMMs) have come to the fore, but maybe they have done so a little too much.
An MMM will try to draw a conclusion between correlating data (often using regression modelling or Bayesian inference) which as discussed earlier can have problems. Theoretically these models work, but in practice the level of data required for meaningful, rigorous conclusions is often not available.
In contrast to an MMM which can be a victim of its own complexity, the scientific method uses simple models that hold under a wide range of conditions and can be verified experimentally.
To measure a campaign scientifically, advertisers need to use single-source data, exposed groups, and control groups.
In practice this means isolating a group of people based on whether or not they saw a certain ad, and comparing their behaviour with a group of people with the same attributes who didn’t see the ad to then accurately measure its impact.
Some media owners are now starting to offer this type of measurement.
The scientific method is the most powerful method of acquiring knowledge humankind has ever had, and marketers should not forget this.
Welcome the new guard
Metrics can sway behaviours and swing outcomes for the better or worse, particularly in the age of AI and algorithms. Armed with that knowledge, advertisers should take the time to figure out what is actually working and embrace the new metrics that are now available to them.
Relying on the vanity metrics of the past is a trap, and those metrics have been used as a justification for the existence of ineffective marketing for too long.
FY24 budgets will be increasingly scrutinised and therefore prioritising metrics that deliver sustainable, profitable sales growth in the short and long term will lead to bigger budgets and better results for marketers.
Liam Shaw is the co-chief executive officer of the vehicle-based OOH service Wrappr.
I can see how measuring the impact of Wrappr in the context of other media would be challenging and frustrating, but to claim REACH as a vanity metric does to my mind somewhat undermine the rest of the article.
User ID not verified.
Yep, you don’t want to touch that stuff!
Hey Alex, I’ve got nothing against reach as long as it’s acknowledged what its limitations are, and it’s not being relied on when there are much better ways for demonstrating campaign effectiveness. I see how the second paragraph may have read like I was dismissing it entirely, which wasn’t the intention. Appreciate you calling it out.
Always have to be careful with taking things from overseas- hat single source data are you discussing? The single source that is at an always on scale that I know of that you could use for this in other markets is in fact fused data in Australia. Which means it’s technically not single source as it uses the horoscopes version of data matching to say people are the same and match purchases with media consumption. That’s definitely not scientific- it’s in fact worse.
User ID not verified.