Social media monitors (SMMs) trawl the web to find mentions of your brand or what ever it is that you’re interested in monitoring.
There are many SMM products and services available: some free, some you pay for.
Here’s a very basic example;
http://www.whostalkin.com/
If you type in the name of a brand or topic of interest, you’ll get an idea of the kind of information SMMs return.
Depending on the level of sophistication built into the SMM you use, you can refine your search with key words, run analytics, see where the buzz is happening etc.
There’s a lot of hype around SMMs. Not surprising really. The idea – getting feedback on the cyber-buzz around your brand, product or service – is timely and sounds quite marvelous!
Kind of. Until you think about it a bit more. Which I have. And wearing my qualitative researcher’s hat, SMMs actually fail in two important ways;
1. Sample definition
2. Sentiment
Sample definition
What constitutes a SMM sample? In a nutshell, a SMM sample comprises the searchable/findable content sourced from various online channels. That’s as precise as you can get really. The truth is, you just can’t know who’s represented (or not) within that content.
For example, SMMs can’t identify and screen out marketing blogs, websites or chatter. This means that SMMs don’t distinguish between content generated by marketing folk and content generated by non-marketing folk.
And let’s face it, quite a lot (most?) of the brand chatter out there is actually generated, nurtured and sent bouncing around the interwebs by marketing folk. People like us. The kind of people we try very hard to screen out of market research samples.
Also worth noting is that SMMs can’t distinguish between content generated by core customers, infrequent customers or non-customers. This means that all customer/brand relationship variations are automatically given the same share of voice and weight in the analysis.
Another factor to consider is that the sample will be skewed. And while a sample skew, in itself, is not necessarily a problem, it’s certainly a problem when you don’t know how it’s skewed. Which is the case here.
Without being able to define the sample, and without knowing how the sample is skewed, there’s no foundation or context for meaningful content analysis.
Next time, I’ll take a look at sentiment…
Filed under: market research | 10 Comments
-
-
-
Click here to subscribe with RSS
-
-
-
Zebra Bites by Katie Harris is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 2.5 Australia License. -
Great post Katie- looking forward to hearing what you’ve got for sentiment! I’ve been observing Matt Granfield’s work with dp dialogue (http://www.dpdialogue.com.au/). They’ve come up with a tool which monitors SMM activity but also sentiment attached to this activity. In the end, you can see who are your SMM fans, hecklers and fence sitters – very interesting but probably something for post 2.
Hi Katie, I’ve been through the same thought process as you quite a number of times over the last year and the result, as Nathan above me suggested, is our social media monitoring tool, Dialogix (http://www.dpdialogue.com.au/dialogix.html). It does in fact contextualise conversations, segment them and rank their sentiment. It knows the difference between a marketer and an average blogger (even if they’re not disclosing they’re a blogger but happen to share an IP address with a marketing firm). Take a peek if you like – I can give you a free trial if you’re keen!
Hi Nathan and Matt
…bet you picked up this post via a SMM. : P
Matt – I’d love to see Dialogix in action!
In the meantime, questions I need to be able to answer for my clients;
Does it identify marketers’ – including those who work in PR, market research, advertising etc – *personal* blogs? For example, would it automatically identify Zebra Bites as a marketing blog even though it sits on WordPress (not on our Zebra website)?
Would it identify me as a marketer when I leave a comment on someone else’s blog (even though I don’t post from Zebra’s IP)? Or does it only identify marketers that post from IP addresses you’ve (manually??) identified as marketing firms?
Still not sure about how one would go about tackling the broader sample definition and sample skew issues…
Absolutely, especially if your client is a major brand, you could spend all day filtering through the hundreds of conversations that appear.
As Matt says, there are various paid for services out there that do the filtering for you, as well as his, one which we recently had a demo of and which seems to do the job pretty well is Radian 6 (radian6.com)
Hi Dirk and gang,
Thanks for the mention of Radian6, and hope you enjoyed the demo.
Sentiment in social media monitoring is always going to require a very important filter: humans. It’s very much similar to the idea that when you get comments on your product through your website, or read reviews in a trade journal, or get customer service phone calls. Having company representatives on the other end of the information stream in order to evaluate the information is key.
The Radian6 platform excels at gathering and sorting massive amounts of information into digestible pieces. And our customers are far and away more efficient because they can focus not on gathering and sorting the information, but analyzing it and determining the best way to put it to use to move their business forward.
We’re always striving to make the process of monitoring social media as seamless as possible, and really appreciate discussions like yours that outline both the challenges and opportunities in front of us. Thanks again.
Best,
Amber Naslund
Director of Community | Radian6
@AmberCadabra
Hi Dirk and Amber
Thank you both for your comments. It’s such an interesting topic for me.
Will be posting some thoughts on sentiment soon; would be interested in your comments on that too!
: )
Does the fact that Nathan and I found your post before Radian6 make our tool better? Nope. It means we’re better human analysts, which is the key to it all. You can build the best filter in the world and still make major errors. Robots are awful at contextualising dialogue, but they are excellent at learning patterns of behaviour and data. Dialogix learns well and it has a smart team of humans contextualising the information it finds before our clients see it. We’re still in beta phase at the moment, but I’ll add you as a tester in the next round if you like!
Matt – couldn’t agree with you more. Human analysis most definitely holds the key.
Now if we could just sort the sample and skew issues so that we can focus that analysis appropriately…
; P
And yes please! Would absolutely love to be a tester in the next round.
Hi Katie,
Problem yes.
This is similar to the debates:
(1) smaller response rates: Are they representative? You don’t know if you don’t have something to compare it to, which is a major problem for many businesses, especially SMEs.
(2) online v paper: Which will capture the most? This doesn’t allow for why the non-participants are not participating: the technology issue.
(3) incentive v no incentive: Does it bias the participants? Or, is the focus just on increasing the number of participants?
SMM thoughts:
(1) SMMs are able to collect large quantities of data but you don’t know enough about the participants to compare to a database, if you had one.
(2) SMMs are capturing buzz online but it is also only capturing those that are loud and participating. Do the non participating readers have the same views?
(3) Do those creating buzz have a tribe? What is the incentive for people to create the buzz?
Not knowing information about the buzz creators (other than marketers being excluded) is not too big an issue if that is known, considered and methodology developed to manage it. It is showing a trend that needs to be further investigated rather than being acted on independently.
In a way you can define the sample: online and communicating in that environment. It isn’t perfect, but perfection is rare (non-existent). We could get into a whole quant v qual and survey design debate there. It is better than not being open to the views out there. I often talk to business owners who don’t “ask” anything from their tribe and therefore miss the opportunity to gain their ideas in any form.
You need to follow up the buzz information with a survey of your tribe, the active and non-active ones to see if they have similar views. It gives a great resource for developing a quantitative survey to understand the buzz at a deeper level.
Kate.
Hi Kate
Thank you for this. Great thinking as always, and a lot of quality food for thought.
I think the key thing, as you point out, is that these issues are;
- acknowledged
- given appropriate consideration, and
- that methodologies are developed to manage them
I can see great value in having a quantitatively/research design savvy consultant involved in the process and in making sense of the output.