I recently answered a few questions about market research for Marketing Magazine. I must say, it was nice to be at the other end of the questions!

I think the article only appeared in the print edition (June, 2009), so over the next few days I’m going to post the nutshell version of the questions (4 altogether) and my answers.

Please feel free to add your own thoughts!

Q1: How do marketers get the best value out of market research when their budgets are being diminished?

A: Marketers will get the best value out of market research by making sure they do 3 things;

1. Ask good questions (ie have clear and well defined objectives)
2. Ask the right people the good questions (ie employ a useful sample)
3. Ask the right people the good questions in the right forum (ie choose the optimal methodology)

Sounds simple, but this is where the clever thinking is. If you get these 3 right, you’ll get useful data, efficiently.

I’ve always been a big fan of desk research and in particular, the literature review.

Even in the days when it meant searching the CD-Rom database at the UNSW library (any one remember Psych-Lit?) to find the ‘literature’.

I’d scribble down the references on a scrap of paper and go, on foot, to hunt for the hard copies. These would invariably be housed up on the fifth floor, down the very end, where the heaters didn’t work in winter. But it didn’t diminish the thrill of the ‘find’.

I danced with (some might say nerd-like) delight when it became possible to access the library catalogue via the internet (albeit in the early days, you could still only get information on the item’s location, not the actual item – but it was still exciting!).

Things have changed considerably since then.

From the way we source information, to the type of information we end up with. All changed.

But importantly, it’s not just the content that’s changed, it’s probably also the quality.

Why? Because some information is (a lot!) easier to get than other information.

I sometimes wonder whether we’re too quick to stop at the easy-to-get-to stuff.

Do we know when and how to dig further for better information?

Are we teaching the next generation of researchers how to do this?

A skill worth honing

Being able to weave a literature review together is a skill worth learning and/or a skill worth honing, particularly for researchers.

Beyond the likelihood of increasing the quality of information one ends up with, the process can be enormously enlightening.

It provides a feel for the breadth, and often surprising depth, of knowledge around any particular subject; inspiring and humbling at the same time.

It provides exposure to confronting, yet compelling views, often contrary to one’s own.

It’s a way of learning what wheat looks like vs chaff.

More context. Better information. Better research.

Busy times at Zebra, so just a snack-sized bite…

Question: What’s the difference between good market research and bad market research?

Answer: Good market research makes you money; bad market research costs you money.

I think a key part of our role, as professional market researchers, is to advise and steer our clients on, and towards, the research methodologies that will effectively and efficiently answer their research questions.

“Yeah, and…?” I hear you mutter. “You’re stating the bleeding obvious”.

If it is the bleeding obvious, then I’m confused.

Because while I busy myself with answering that brief, the passionate embrace of all things 2.0 (for the want of a better descriptor) by some researchers – along with the often alarming and dire warnings for the future of Research 1.0 (for the want of a better descriptor) – suggest that marketers must have suddenly changed their questions.

Have they?

In some cases, yes. The world itself has changed/is changing, and the marketing context is changing too. But it’s not changing entirely, and importantly, it’s not always changing in parallel.

A Research 2.0 solution suggests that marketers’ questions have changed as quickly and as radically as online technologies and forums themselves.

But marketers still want to know how to sell stuff. Does Research 2.0 output help them do that? Does the information gleaned from the ‘new’ listening posts answer their fundamental market research questions?

Or are those at the helm of the Research 2.0 front actually changing the brief itself? Shaping the research questions to fit the new technologies?

Is it research?

Perhaps what we’re defining as Research 2.0 isn’t, in fact, market research at all. Maybe it sits outside the realm of market research; more in the customer relations/customer service domain.

Without doubt, the online environment provides marketers with invaluable feedback – but of a very specific kind. Quite clearly, because of its skews and tip of the tailness, it’s not the kind that’s of much use in making big-marketing-budget-decisions.

Maybe, as market researchers, we are in pole position to harness and distill that specific feedback for what it may be worth. We can certainly lend our experience and caution to the analysis.

But are Research 2.0 methodologies really the silver bullet they’re being sold as? Is it a marriage of the right questions, with the right sample, in the right context?

Or is it a shotgun wedding?

So after a slight diversion, it’s back to finish off my thoughts on advertising concept research.

I’ve talked about the importance of capturing un-considered responses and this is my main issue with researching advertising concepts online. There’s simply too much time for respondents to think.

Between the time you show them the advertising, and the time it takes to read and process the question, they’re going to have time to think. And then they’ll have (unlimited) time to craft their responses. And edit them. And polish them.

In effect, responses are likely to be sanitised beyond usefulness. They won’t give you any sense of whether, and/or how, the advertising is actually working or not.

Tools

There’s a whole raft of tools available for researchers that purport to enable online advertising concept research. These tools allow ‘participants’ to tag or mark up the stimulus, eg proposed copy, website etc.

I can see why this idea has appeal; within the co-creation paradigm, it scores a 9/10, right?

Yes, it does. A wonderful tool to use if you want to involve your geographically-dispersed design team. But you’ll struggle to get anything useful from your non-design-schooled research participants.

Why? Because the focus will be on the stimulus, not the concept.

The ‘design’ tweaks respondents make will be just that; design tweaks. And are they really going to do a better job than your design team?

Without an opportunity to explore respondents’ reasons for their tags and mark ups, red herrings are all you’ll be eating for dinner.

Maybe I should stick to knitting, but despite my obvious leaning towards qualitative (vs quantitative) research, I feel compelled to write about a quantitative research issue that isn’t quite getting the consideration it should; sample representativeness. This, I might add, seems to be a problem particularly – although by no means exclusively – for quantitative research conducted in the online environment.

Sample representativeness

When we (when I say ‘we’, I don’t mean you or me of course, I mean them, but let’s go with ‘we’) undertake a quantitative market research study, it’s rare that we have either the time, or a budget, that would allow us talk to each and every customer/potential customer of interest.

Instead, we choose a selection of those customers/potential customers to represent the greater population of interest to us. In research-speak, this selection is called the sample.

In a quantitative research context, the way you choose your sample, and the structure of that sample, is everything. These two factors will pretty much define the extent to which you can extrapolate your research findings to the population of interest. In non-research speak, that means the extent to which you can have any confidence in the research results.

Making sure that the sample you want to use to generalise to the greater population of interest is representative of that greater population is;

1. Of vital importance

2. Not always easy

3. Of vital importance

Number 3 isn’t a typo. Issues arising from number 2 often mean that the sample may be seriously compromised. I put number 3 there as a reminder.

It’s almost 5 months to the day since the first Side of Porcupines * post; definitely time for another one!

The criteria is the same as before, but just to highlight the 3 most important things;

1. You have to have written it,
2. The post doesn’t have to have attracted hits/comments (but it’s ok if it did),
3. You just have to like it and/or want to share it.

Link away!

: )

* An overview of Porcupine reads

Social media monitors (SMMs) trawl the web to find mentions of your brand or what ever it is that you’re interested in monitoring.

There are many SMM products and services available: some free, some you pay for.

Here’s a very basic example;
http://www.whostalkin.com/

If you type in the name of a brand or topic of interest, you’ll get an idea of the kind of information SMMs return.

Depending on the level of sophistication built into the SMM you use, you can refine your search with key words, run analytics, see where the buzz is happening etc.

There’s a lot of hype around SMMs. Not surprising really. The idea – getting feedback on the cyber-buzz around your brand, product or service – is timely and sounds quite marvelous!

Kind of. Until you think about it a bit more. Which I have. And wearing my qualitative researcher’s hat, SMMs actually fail in two important ways;

1. Sample definition
2. Sentiment

Sample definition

What constitutes a SMM sample? In a nutshell, a SMM sample comprises the searchable/findable content sourced from various online channels. That’s as precise as you can get really. The truth is, you just can’t know who’s represented (or not) within that content.

For example, SMMs can’t identify and screen out marketing blogs, websites or chatter. This means that SMMs don’t distinguish between content generated by marketing folk and content generated by non-marketing folk.

And let’s face it, quite a lot (most?) of the brand chatter out there is actually generated, nurtured and sent bouncing around the interwebs by marketing folk. People like us. The kind of people we try very hard to screen out of market research samples.

Also worth noting is that SMMs can’t distinguish between content generated by core customers, infrequent customers or non-customers. This means that all customer/brand relationship variations are automatically given the same share of voice and weight in the analysis.

Another factor to consider is that the sample will be skewed. And while a sample skew, in itself, is not necessarily a problem, it’s certainly a problem when you don’t know how it’s skewed. Which is the case here.

Without being able to define the sample, and without knowing how the sample is skewed, there’s no foundation or context for meaningful content analysis.

Next time, I’ll take a look at sentiment…

It’s the time of year for the Zebras to make way for reindeer (or the Six White Boomers).

We’re still working, like little elves behind the scenes, so if you need to contact us, please do. The blog, however, is taking a break until early in the New Year.

And on that note, wishing you all a very happy and healthy 2009!

I think I’ve found my blog-soul mate.

Nice to find someone who shares my passion for user friendly research.

*Just in case* you missed my rather excited tweets or Facebook update, here’s my opinion piece in this issue of Anthill magazine.

: P

The seedling is here.