The pursuit of effectiveness

Some opinion on measuring marketing effectiveness, of course, is extremely valuable and has done much to move marketing forward, but some of it is, well, less so. But because until now there has been little by way of codified learning to help us all discriminate between the valuable and the fallible, we have remained at the mercy of opinion. Small wonder then, that an army of management consultants and market research companies are able to impose their own models, however bizarre, upon the marketing world.

The greater problem is that under the weight of all this opinion, the difference between hypothesis and fact has become blurred. So small wonder too, that most CEOs regard the world of marketing and communications as 'undisciplined' and 'uncommercial', according to a McKinsey study. And herein lies the source of a further problem.

ACCOUNTABILITY AND EFFECTIVENESS

Not unreasonably, general management has imposed an ever-increasing burden of accountability upon this rebel world, as it seeks to bring it to heel. But the trouble with imposing accountability on marketing when there are so many conflicting and questionable theories about what drives effectiveness, is that it inevitably leads to the wrong behaviours.

The analysis reported here shows that the evaluation is widely flawed, the metrics are usually the wrong ones and the results too often lead to the destruction of shareholder value. In the pursuit of accountability (being seen to do something) a conflict arises with effectiveness (doing the right thing).

MINING THE IPA DATABANK

So we have attempted to do our bit to create an alternative to opinion: an objective and rigorous analysis of the 880 case studies of effectiveness residing in the IPA Effectiveness data-BANK. The data-BANK is probably the world's largest database on effectiveness and is based on entries to the IPA effectiveness awards, widely accepted as the world's most rigorous effectiveness competition.

The data-BANK includes a lot of confidential additional information that is not included in the published awards entries. This covers many details of the trading environment of the brand, of the status of the brand, of the objectives and features of the campaign and of its results (both business effects and intermediate consumer effects). Thanks to this additional data we are able to look at what actually works in business terms – which has nothing to do with what wins prizes.

Our key measure of effectiveness is what we have called the 'Effectiveness Success Rate'. This is the percentage of cases that report 'very large' effects in any of a number of key business metrics, such as profit, market share, price elasticity etc. This measure turns out to be closely related to marketing payback: the higher the 'Effectiveness Success Rate', the higher the ROI. And unlike actual profit data (which is not widely available), this measure is available for hundreds of cases, allowing us to perform detailed analysis of what drives business success.

The findings of our analysis are published by the IPA in their data-MINE series under the title Marketing in the Era of Accountability. Those findings reveal the dangers of too much opinion – that much accepted wisdom is wrong and much common practice is inefficient. So here we summarise the key areas where common practice and best practice diverge.

1 BE CAREFUL WHAT YOU WISH FOR

The data-BANK reveals that the choice of objectives and therefore metrics is crucial to effectiveness. Choosing the wrong metrics will very significantly weaken the ability of marketing to create shareholder value.

The Problem with Single Campaign Objectives

It is common to focus on a single campaign objective. Sometimes this will be a 'hard' business or behavioural objective (such as sales growth or customer loyalty), but more often it will be a 'soft' intermediate consumer objective (such as brand awareness or image).

Accepted wisdom is that having a tight focus is good, because it will make progress more likely. Unfortunately campaigns with single objectives are dramatically less effective than ones with multiple objectives.

And there is good reason for this. There are many 'levers' of profitability for most brands (such as sales volume, market share and price sensitivity) and the more of these that the campaign 'pulls' the more profitable it will be. Consequently the data-BANK shows that the more shifts there are across a wide range of metrics, the more profitable the campaign will be. The relationship between the number of different metrics that shift and the Effectiveness Success Rate is strikingly linear (Figure 1).

A Balanced Scorecard?

There is a clear implication of this: that brands should adopt a 'balanced scorecard' approach with multiple brand metrics, rather than using any single metric, however seductive the opinion lying behind it may be.

As well as suggesting a balanced scorecard, with multiple KPIs, the data-BANK also tells us that they should be prioritised as follows:

1. hard business metrics (e.g. share)

2. hard behavioural metrics (e.g. penetration)

3. intermediate consumer metrics (e.g. awareness).

And the reason for this is that hard metrics are much better predictors of profitability than soft metrics.

In practice though, intermediate consumer measures are the most commonly used and are often given supremacy.  Intermediate measures are seductive as a focus for marketing, because they tend to move more quickly and impressively, and are easier to link to marketing activity. They are therefore widely used as leading indicators and far too much attention is paid to them in establishing accountability.

Worse still, the focus tends to be on a narrow range of intermediate measures, particularly awareness. And worse even than this, the Data-BANK shows that the commonest leading indicators (brand awareness, image and advertising standout) are in fact some of the poorest predictors of business success.

But that is not the end of it. Even when marketing is focused on business metrics it tends to be focused on the wrong ones: sales rather than market share or profit, volume rather than value, and customer loyalty rather than penetration. And the focus is almost never on price sensitivity as a KPI. Yet the data-BANK demonstrates that brands that focus on profit, value share, penetration or price sensitivity dramatically outperform those that pursue the common goals. Again, the data suggests that this tendency is at least partly driven by accountability and the consequent pressure to choose measures that demonstrate progress most easily.

It is worth dwelling on penetration and price sensitivity for a moment, since they are both powerful routes to business success.

Loyalty

We have all been influenced by the work on loyalty of Bain and McKinsey over the years. Not least the famous Bain thought experiment

(that is in fact what it was) that a 5% improvement in customer retention could result in a 25–85% improvement in profitability. The CRM movement has in part been founded on this line of thinking. We don't doubt the truth of the pronouncement as an observation on accounting realities, just as we don't doubt that being able to turn lead into gold would make us very rich men. But vast amounts of work by Professor Ehrenberg, as well as the evidence from the data-BANK, suggests that loyalty is almost as elusive as alchemists' gold. Ehrenberg has demonstrated that loyalty, defined objectively as share of category requirements over multiple purchase cycles, is pretty much constant within a category, and is only influenced by the size of the brand. He has also suggested that marketing's most productive goal is therefore to defend or build penetration. The IPA data-BANK strongly supports this point of view: effectiveness success rates are dramatically higher for campaigns that aim to increase penetration than for those that aim to increase loyalty. Pursuing both (i.e. talking to customers and non-customers) is even more productive, but pursuing loyalty alone is a recipe for underachievement (Figure 2).

Again there are good reasons behind the findings: usually non-customers are much more numerous than customers and customers are much more influenced by experience of the brand than marketing communications. We are not saying don't try to keep customers happy.

In the era of internet driven word of mouth, happy customers are an important recruitment tool for new customers. And that is why targeting both is so productive – not because marketing can suddenly turn occasional users into most-often ones.

Price Sensitivity

Price sensitivity, on the other hand, is an important metric. It is difficult to measure reliably without econometrics, which perhaps explains why only 4% of the cases in the data-BANK targeted it. Yet they enjoyed an extremely high effectiveness success rate (83%). In our view price sensitivity is a much more useful measure of customers' commitment to a brand than conventional measures of loyalty, because it is something that can be influenced by marketing, and has a powerful impact on profitability. Again there is good reason behind the finding.

The benefits of firmer pricing fall entirely to the bottom line, whereas volume increases bring costs with them.

Pretesting

There is one final area where a commonly used metric can backfire badly. It is increasingly common practice to pre-test advertising quantitatively in order to attempt to predict its likely business success in-market. A common metric of such pre-tests is predicted advertising standout. We have already demonstrated that standout is a highly fallible predictor of business success: so pre-testing is providing a fallible prediction of a highly fallible metric. It might be expected to be unreliable and the evidence of the data-BANK is that it is not only unreliable, but probably actually reduces effectiveness (Figure 3). But accountability continues to mandate it.

By contrast econometric modelling is widely misunderstood in marketing and consequently rarely used. Yet the data-BANK shows that it is an extremely powerful planning tool that if used on an on-going basis can improve the profitability of brands. It does this by giving management the information needed to operate the many levers of brand profitability optimally.

2 USE YOUR HEAD – LISTEN TO YOUR HEART

The various campaigns reported in the data-BANK can be grouped into 3 types: principally rational (largely involving the communication of information for effect), principally emotional (largely involving the creation of emotional engagement for effect) and those that do both in roughly equal measure. Rational campaigns are more common than emotional ones. Exactly why this is so, when there has been so much new learning published in recent years about the superior effectiveness of emotional communications, is unclear. There is a suggestion in the data-BANK that it may be because rational campaigns are more easy to measure in terms of leading (intermediate) indicators – and hence appeal to accountability-minded folk. What is clear from the data-BANK is that emotional campaigns are considerably more effective – and in particular more profitable – than rational campaigns. And combined emotional/rational campaigns do not give you the best of both worlds, they give you a mushy average (Figure 4).

So emotional engagement is good for effectiveness – hardly front-page news, but hopefully we have helped move it from opinion to fact.

But the data-BANK classification of campaigns is richer than this, and when we look at greater depth it emerges that one broad emotional communications strategy is more successful than any other: 'fame'.

Brand Buzz

The terminology needs careful definition here. Brand fame is not the same as brand awareness (which we saw earlier does not reliably lead to business success): it is about creating 'conversation' and buzz around the brand – giving it the sense of being the brand that is making waves in its category, and the authority that comes with that. Thus fame builds a broad sense of brand health by creating perceptions that a brand is widely valued, whereas awareness merely creates knowledge of its presence.

It is not surprising that fame is so much more valuable than simple awareness – brands long since ceased to be such simple entities as 'reminders of existence'. In the modern world they are highly complex and the best ones are highly emotionally charged. Fame campaigns tend to generate more intermediate effects, more business effects, and produce bigger paybacks. The fame approach is therefore both easier to measure (good for accountability) and more effective: it deserves to be much more popular than it is.

3 GET REAL

Having the right strategy and the right creative is not enough. You also need the right budget. The data-BANK shows that there is a very clear relationship between budget levels and effectiveness. In particular, the relationship between share of market (SOM) and share of voice (SOV) turns out to be crucial.

Once again, common practice in this area is not the same as best practice. Most brands tend to set their share of voice roughly equal to their market share, and expect this to deliver growth. But our analysis shows that this is not enough. If you want your brand to grow, you need to set your share of voice above your market share.

We call the difference between SOV and SOM the 'excess' share of voice, and it is this excess SOV that drives growth. In fact, the data suggests that, for a typical brand, a 1% increase in market share requires around 10% excess SOV. This means that strong growth usually requires a high advertising-to-sales ratio – much higher than many marketers seem to realise.

4 DON'T BECOME A FASHION VICTIM

It has become fashionable to opine that TV is a dying medium in the digital era, whose effectiveness must surely be ebbing away. This is despite the fact that a great many of the world's largest and most savvy marketers continue to invest heavily in TV. The data-BANK shows that, not only is TV an extremely effective medium, but also that its effectiveness is growing over time (Figure 5). Again there is good reason behind the finding. Growing competition has driven real TV costs down to a 25-year low, whilst, contrary to popular opinion, commercial TV viewing has in fact been rising. Add this to the supremacy of TV as an emotional medium and you have the ingredients of its growing effectiveness.

There is also a fashion for 'surround-sound' advertising – the bombardment of consumers with advertising touch-points wherever they are. The databank shows that integrated multi-channel campaigns are very good for effectiveness, and that adding more non-advertising channels to the mix is also good, but that the same is not true of advertising media. The reason for this is that non-advertising media are usually used differently and are therefore additive. Merely repeating the same message across media inevitably leads to diminishing returns.

5 ROI – A LOOSE CANON?

ROI is a fundamental canon of accountability and it is the common yardstick of payback. The data-BANK argues strongly for a greater focus on payback – especially in the light of a recent IPA Bell-weather finding that only 53% of marketers formally track financial return. But we do not advocate focussing on ROI as the metric.

ROI is a potentially dangerous way to measure payback in the marketing context, because one very easy way to boost ROI in the short term is to cut the marketing budget. In the longer term of course, this will result in the destruction of shareholder value. Moreover, most use of ROI is confused and it is very rarely calculated accurately, so it leads to poor business decisions. So we recommend a better way of measuring payback that will circumvent this potential problem.

CONCLUSION

Our objective has been to use the data-BANK to test the wisdom of a lot of widely held opinion about marketing – and much of it has been shown to be wrong. What is more worrying is the consequent impact on effectiveness when growing accountability pressures are overlaid.

If accountability in the shape of value based remuneration is to deliver more effective marketing, it must be based on sound metrics. We urge marketing leaders to discriminate more carefully in future between opinion and fact. Keep both eyes open.

Figure 1: The more metrics that shift, the greater the effectiveness

Figure 2: The myth of loyalty

Figure 3: The effect of pre-testing on effectiveness

Figure 4: The profitability of emotional vs. rational campaigns

Figure 5: The growing effectiveness of TV