It’s tough being in market research at the moment, especially in the UK, where the general election earlier this month showed how difficult political opinion research can be. No surveys predicted the outcome (though constituency-based, first-past-the-post electoral systems add a layer of complexity to an already difficult type of research).
Everywhere, the relentless demand for numbers-based news stories and data to underpin decision making has driven the market research industry to develop lower-cost online and social media approaches to research, and in some cases this can compromise the integrity of results. Using self-selecting online panels, members of which may be paid (small sums) and who answer dozens of surveys as quickly as possible, means the design of the survey, and the sampling, needs very careful thought – nothing too complex; nothing in depth; and careful analysis by subsample. And of course, the availability of such approaches to users directly, without the mediation of a specialist agency, removes a layer of quality control.
Simultaneously, the use of conventional market research techniques to measure actions (“Which of the following xxxx have you done in the last three months?”) is being replaced by “big data” approaches that can analyse transactions directly (from online sources), without having to reply on humans’ imperfect memories, or willingness to disclose their activity, and where huge numbers of correlations may be tested.
Do the fundamentals of research need rethinking?
And yet, while they are significant, these issues are perhaps the least interesting of the challenges facing market researchers. What’s more intriguing is the possibility that we should drop our assumption that the respondent is rational. Not just on questions of political allegiance, but on brand allegiance, or buying behaviour, or pretty much any question you might think of asking. This would also apply where big transaction data can’t help very much, such as in market testing of a new product, say, in order to complete the revenue line in a business plan. How can you be sure that, say, 35% of your target customers will buy it just because 35% of them told you they would?
In fact, this type of market research has only ever been at best an indication of future behaviour. One can, and should, be much more sophisticated – for instance, trading off attributes of a new product or service against one another, or against price, or against a competing product or service with another brand: more engaging for the respondent; more valuable for the company asking the question.
Market research can’t generate new ideas
One thing Innovation Observatory found a long time ago is that the use of market research in the innovation process is quite difficult. Ask a group of consumers what new things they would like, and they generally don’t come up with anything original – it tends to be minor variations of what they already have – more of this; less of that; cheaper for the same. That’s why the most innovative companies tend not to use market research to generate innovation, but only to test ideas once they’ve been fleshed out a bit.
Think more carefully in designing research
So what’s to be done? I think there are two paths – both valid. The first is to be cleverer earlier in the process. Think more deeply about the purpose of research. Think about the choice of techniques (there are more of them than ever before) and don’t set unrealistic budgets. (If the only techniques you can afford won’t deliver you results you can believe, it’s wasted money.)
The second is to be simpler and recognise the limitations of all research, and particularly of the research you have chosen to carry out. It may be that research results are actually most useful as the beginning of a process – not the end: something to start a discussion, or to fire up creativity.