How do I decide on which buzz monitoring tools to use?

My most popular blog post in the last year has comfortably been the post on buzz monitoring software tools (aka social listening tools). Problem is, there are around 30 tools on the list and of course, there are more not covered.

So the obvious question for the marketer is "how on earth do I decide which tool to select for my company?". Well the original post does categorise the software according to purpose, but selecting the right one needs some assistance.

I had seen some nice case studies about a tool called Social Radar, including this one on the Infegy Buzz Study blog using it to evaluate sentiment around the iPad launch. So when I got a chance to meet and interview Gray Dudek, the Managing Director of Infegy Europe, the creators of Social Radar, I had some obvious questions to ask, and I asked them from a potential customers point-of-view, but not starting with price, there are some other questions which should be asked first.

Here are Gray's answers...

Your Goals

Q1. What are we trying to achieve with this software

[Gray Dudek, Infegy] Deciding on your goals and/or requirements is, of course, vital  when looking to identify suppliers for buzz monitoring, social media measurement, chatter analytics, the list goes on regards terminology.

Goals are very different to requirements as goals are visionary and requirements, I would say, are mandatory, think of them as "I would like" vs. "I definitely need" respectively.

But before you can get that far it's important to understand the bigger question.

Most treading this path will eventually have to answer the following first:

"Is the business, brand or service provider I am doing this for structured in such a way that this is a simple or relatively difficult task?"

An oddly vague question with possibly a mysteriously answer so let's break it down a little further.

What I mean by that is does the business  already underpin everything it does via metrics (clicks, EAV's, share of voice, etc) or does the business simply take gut instinct and past experience then build from there? The two are very different and identifying which position the business is in will drive your methodology for defining and deciding on your goals and requirements and ultimately how you will sell in the concept to the board.

So, if your goal is to measure existing chatter to define strategies for the future then a system that can go back in time, for want of better words, over at least 12 months will suit your needs as it can be utilised for planning based on past campaigns. This type of system will also support the sort of business that is metric-orientated as synergies can be drawn across past campaigns, schedules, etc.

However, if your goal is customer service orientated and your business focuses on past experience and gut instinct then a tool that allows you to respond to online chatter within the system will suit better in this instance.

Defining your business position, thinking method and goals (in that order) will serve you well when it comes to defining the list of providers.

How do we identify the main contenders?

Q2. How should I long and short list?

Long and short listing is somewhat difficult at present due to the varying methods that providers use to price their systems. Some use all in prices, some use component-based prices and some decide their price based upon your volume/level of use. All have their relative merits but some more so than others. I'll cover these later.

So back to listing. Having defined your business position, thinking method and goals you should build a short document that you can send to providers that gives them the opportunity to respond defining how their systems and tools can support your needs. So often providers are asked for product demo's only to find out later in the same week that the system they presented simply will not fit the needs as it cannot satisfy the goals. The document, call it a brief if you will, allows you to save time by removing organically from the long list those providers who will step back from the conversation due to incompatibility.

The next step is to request product demos (not test accounts!) from the list you now have. At this stage I would not suggest asking for a test account think of it like jumping into a pool before learning to doggie paddle out of the water: you're probably not going to have a good experience!

Sounds obvious, but all systems rely heavily on the user knowing how to get the best from them by means of query language, data mapping, etc, etc. This sounds a little difficult but trust me, it's not; it's about learning the basics and building from there.

I digress to the topic of User Training and Service from Providers but that's one for a later date.

Criteria to review in demo

Going back to your original brief will help here. All the systems are pretty clever in what they do. Some will graph chatter versus geography (I'm dubious about the tech used for this but hey, it looks pretty). Some will measure of the sentiment of chatter ie. what's positive and what's negative. Some will enable you to respond to Tweets and posts in-tool as opposed to ducking out the web to react on the site in question and so on.

But which of these do you actually need to achieve your goals? There is a cost associated with all these clever functions and if you don't need them you will end up paying for them and not using them just like many of us have done with make your own ice cream machines!  Clever idea, bought one and now it's in the cupboard gathering dust.

How do we compare buzz monitoring solutions?

Q3. Which criteria do we use?

There are many questions that will come to mind when reviewing the systems but the foundation questions to ask in my opinion, as shared by Brad Little from Nielsen, are:

1. What's included in the data sets?

Does it cover blogs, Tweets, reviews, forums, etc

2. Where is the content coming from and how is it harvested?
Is the data taken from 3^rd party providers or is the data collected in-house. An important point to consider if the provider offers the ability to go back in time over and above 6 months as data is regularly dropped out by 3^rd party providers and thus provides a diluted view of days gone by.

3. How is the data cleaned and prepared?
What process does the provider go through to manage duplicates, spam, forum threads, etc. You don't want to be double-counting or duplicating responses to customers.

4. How is the data organised or segmented?
Is the remaining content relevant to the business questions being asked? What are the base, volume and discussion sources being included for classification? How is the data being segmented so it contains the most pertinent consumer discussions around your specific area of interest?

5. How is the data being analysed and are actionable insights delivered?
Is sentiment purely done by automated technology or by human analysis, or both? Can the system help you determine what the important topics are that lead volume or drive a particular sentiment? Does the system use an influence index of some sort to identify key persons around sectors/conversations?

6. How actionable are the insights and how should they be implemented?
Is there a consulting service so that information and data can be transformed into insight? CGM is a vast and ever changing form of media and many organisations need to rely on the expertise and experience of a well seasoned research team. While data can be informational, consumer generated media insights are powerful building blocks that can be used to transform and prepare an organisation for the changing digital landscape.

The above will serve you well as base questions to benchmark providers against one another and also to help you understand which of the providers synergise with your original brief.

It's safe to say that you may discover that two systems (or maybe three in some instances) fit different functions of your business or agency due to needs. You may take onboard a provider that focuses on customer engagement and also a provider that focuses on insight & planning. The two are very different and require very different systems to achieve excellent results so plan for this scenario and don't be fooled by the one-size-fits-all comms that some providers put out there!

How much will we have to pay?

Q4. What are the pricing options?

It all comes down to how you want to pay versus what you want to pay. As mentioned before, there are a number of methods being employed by providers and until some form of regulation or market consolidation comes into force this will continue.

You can pay per search, per license, per month, per client and so on. All methods will come out with different prices, some greater than others and some with more limitations than others. Suffice to say that if you want to manage cost effectively (I don't mean "cheap" is this instance) then a fully functioning system with a simple monthly fee is the route to go no hidden costs! If you want to dip in and dip out of using the system then a pay-as-you-go model such as a pay-per-search model will fit you best. It all comes back to your brief and asking the provider, once they have given you a cost, if there is anything else you can pay for over and above what they have quoted. This way you'll find if you are on a base-rate price - the Ryanair model) - there may be surprises around the corner!) or the premium all in service with no nasty month-end costs (the Virgin Atlantic model - everything included up front).