Coincidentally, after my last post, this came up in my Google Reader today;

More on the Problems with Opt-in Internet Surveys

Here’s the first article;

Study Finds Trouble for Opt-in Internet Surveys

I had the great privilege of attending an AMSRS Professional Development session earlier this year to hear Jon Krosnick speak. He was brilliant.

And so it’s with great interest that I follow this very timely and fascinating debate (hosted on Gary Langer’s excellent blog).

Make sure you follow the links to get the full story/debate. It’s an important one.

P.S And here’s a link to the study itself.

So! Continuing with the #stuffiwonder theme…

The telephone vs online survey debate.

The one that goes;

“Really, given that everyone’s moving from landline to mobile/cell these days, telephone survey sample representivity is seriously compromised”.

More often than not (and, of course, depending on who’s doing the debating), it ends with a nod to online panel surveys. In this context, “…they’re probably just as good as – if not better – than telephone surveys”.

Right?

Well, I don’t know.

Panels are opt in. And yes, the same can (and should) be argued about telephone interviews. You most definitely need research participants to opt in beyond a “Bugger off, I’m eating my dinner” response.

But what differences might we see, in terms of motivation and the research output, between a sample comprising individuals who;

• Have been approached randomly (and I get that it’s not really random; the population will be limited to those with landlines), vs
• Sign up to be part of a/several market research panel/s and/to get paid for their opinions?

How do market research online community providers populate their clients’ communities?

(When I say “market research” communities, that’s exactly what I mean; a community used as a market research tool. I’m not talking about online communities that are used in a marketing/customer relations exercise.

I’m not quite sure that the difference is apparent to all, but they’re not the same; not by a long shot.

In one, you’re giving the community members love because you want to make them happy. In the other, the relationship is somewhat more pragmatic; you want to learn from them. Notably, if you’re giving them love to make them happy, you’re not necessarily going to learn much, because they’ll be all nice and lovely back).

Anyway, focusing specifically on market research communities; what checks are in place to ensure that the people who end up in the community represent the people the client actually wants to hear from (ie the population of interest)?

To borrow from the delightful John Lacey, I’m filing this one under #stuffiwonder.