|
Measuring your brand marketing performance is an excellent idea. It allows you to celebrate success, which you must never fail to do in the name of team-building, and, at the other extreme, it allows you to explain away failure, with the added promise of moving things back on track because, as a true professional, you now understand exactly how it all works.
Indeed, arguably, it is more important to have persuasive metrics at your disposal when everything is going wrong, than when success is shaking you by the hand. As they say, failure provides an opportunity to learn, and it will come around from time to time whatever you do, however brilliant you are.
Even better, measuring your performance is getting so much easier, cheaper and quicker. If you are into e-marketing and e-business, the promise is that you can virtually automate measurement from head to toe. If those metrics cross media, programmes and databases, there will be a level of complexity, time, effort and cost involved, but nothing like to the same extent as it would be to do the same thing in the bricks-and-mortar world.
So where is the problem?
The problem is in measuring the right things in the right way, and drawing the right conclusions. Metrics sound simply wonderful until you actually get involved in doing them.
The statistical essence of all metrics is that they must be "valid" and "reliable". "Valid" means that you must actually be measuring what you intend to measure - the things that accurately inform your business decisions. "Reliable" means that the same result must give you the same statistic on each and every occasion that it occurs.
To give a classic old example, the number of babies born in Kent statistically correlates to the number of storks seen flying over Kent. Does that mean that if you count the number of storks, you don't need to count the number of babies? Almost certainly not. They probably both correlate to another issue which might be a much more valid proxy for tracking the number of babies born in Kent.
Anyway, you decide that the only real way of measuring the number of babies born in Kent is to count them, so you ask all the hospitals in the area to provide you with monthly statistics. Unfortunately, some babies are not born in hospitals, and some hospitals forget to complete their returns. Nevertheless, although it is not a valid measure, it may still be reliable because you are using the same methodology each time, so long as the same hospitals remain silent, and so long as the underlying proportion of babies born in hospitals remains the same.
So what is the problem of validity? The biggest one is that the things which are the easiest and most reliable to measure are often not the ones you most need to know about. Take, for instance, trying to find out why your target audience buys your product. If you ask the question openly, you will get the answer price, quality and service. But what do price, quality and service mean? If you apply insightful market research techniques, such as laddering or NLP (neural linguistic programming), you will soon discover that what your respondent really means is that he wants you to help him get promoted, preferably to Board level. That is the valid reality. Now try tracking that in a quantitative survey. "Excuse me, to what extent on a scale of 1-10 does our silicone mastic help you get promoted to Director of your company?" will rapidly elicit "Get out of here!"
If you are involved in social decision making, and you call in a Quality consultant who is used to working on the improvement of production processes in factories, you will soon reach an area of stark mutual incomprehension. In brand marketing, profound insights are still very hard and expensive to get, and exceedingly difficult to track, because the data generally come in statistically small quantities at irregular intervals from intensely variable sources - people.
Trying doing some detailed pricing research using either conjoint or brand price trade-off analyses. In order to set it up, you have to deconstruct the buying process. You have to take into account the way your product category is presented in each store, the different competitors they are stocking, in which product formats, in which quantities, and where in relation to each other. You have to isolate the effects of ephemeral communications campaigns in and out of store that will inevitably distort the results, and make them unrepresentative of next month's market dynamics.
There was a psychologist, whose name I think was Miller, who mocked up a store display. He asked those who participated in his experiment to choose a product from the shelf, and to explain why they had chosen it. Almost everyone chose the same product (because he had fixed it that way through a combination of positioning and design effects), but they all gave completely different reasons as to why they had chosen the same product. In other words, their explanations were in fact random noise. They did not consciously know why they had chosen the product, and that is very often the case. Lesson - do not bother asking people what's important to them; use an experimental or statistical technique to infer importance.
And what is the problem with reliability? In the brand marketing world, are you kidding me? Answer - nothing ever stays the same. The product / service configuration changes, and if yours doesn't a competitor's does. You, or one of your key competitors, suffers a product outage, meaning that you are out of stock for six weeks. You face seasonality effects. There is a downturn in the economy. One of your distributors goes bankrupt. Your major competitor is acquired by another major competitor. One of your regular sources of data suddenly refuses to play ball. You switch research agency, which appears to have a subtle, but fundamental, impact on the data. You decide that it would be far better to collect the data face-to-face than by phone (changes in research methodology always have a significant impact). Your boss decides to cut your research budget, or suggests that you spend it on answering the questions she wants to focus upon. Research International used to argue that they could much more accurately predict the take-up of a new product, with a given level of sales and marketing investment, than the brand manager could predict his budget for next year.
So, what are we saying: don't bother to measure anything?
Well, we once drew up a list of common errors you would encounter if you conducted a piece of market research, from talking to the wrong people, to misunderstanding the question, to errors in recording the answer accurately, to mistakes in data analysis and reporting. From memory we counted more than twenty likely sources of error.
However, we are not saying that you should put your head down, work hard, and devote yourself to seat-of-the-pants marketing and shouting your key messages at your customers until they understand them.
What we are saying is beware of attempting anything too complex (complexity nearly always compounds error), avoid any tracking mechanism you can barely afford (it will more likely end up as a one-off or a few-off), do not take anything on trust (honest people often make mistakes, or see things from only one angle), and keep your wits about you, always asking more questions.
When you are using tools to plan and to measure, insist on simplicity, accept approximate answers (they will be anyway, whether they appear to be or not), and always be willing to challenge them in order to probe the limits of their usefulness.
As with any process, the practice of measurement is a journey more than a destination. We learn far more from discussing the business issues related to measurement with a good researcher, from exploring the strengths and limitations of different methodologies, and from team interaction with, and discussion of, the data, than we usually will from the data themselves. Effective measurement is about teams of people pressing buttons and seeing what happens. Misleading measurement tends to correlate with the employment of ultra-sophisticated black-box techniques, managed by apparent gurus.
So we say: measure for the pleasure of discovering the treasure; do not fall for the fool's gold of shiny technology, inappropriately applied by people who necessarily only understand some of the story.
|
