Posts Tagged ‘measurement’

Think “Over Time,” Not “Point In Time”

ROI-graphA hypothetical scenario for you: your communications director comes to you and tells you that thanks to their department’s activities, there were 200 mentions of your brand online, of which three quarters were positive in tone.

Is that good news? Is it bad?

My answer: I have no idea.

Why? Because there’s no context.

Context, please

As social media begins to mature as a communications opportunity, the pressure to demonstrate measurable results will only increase. However, that measurement needs to have context.

Having three quarters of conversations about you be positve may actually be a bad thing if 80 or 85 per cent are usually positive. Two hundred conversations may actually be a drop in volume compared to the norm. Without context, you have no way of knowing.

Telling me that our online outreach increased the proportion of positive conversations by 15 per cent to 75 per cent means a lot more than just the number alone.

What’s your baseline?

People often talk about social media being a long-term proposition. We need to think about measuring social media in the same terms. That means setting baselines – investing a small amount of effort to draw a line in the sand, from which you can measure your results. Sometimes the baseline may be zero, but in most situations that won’t be the case.

How do you draw that line? Here are a few options:

  • Do a conversation audit – use free or professional listening tools to look at online metrics over a period of time
  • Conduct some market research – commission a few questions in an omnibus poll to measure how things stand
  • Analyze your website statistics – traffic volume, sources, conversions, etc

The way you measure your baseline is up to you. The most important thing is that you do it.

For communicators to justify their budgets, they need to show the delta – i.e. the difference between before and after. Without “the before,” you have nothing.

Make sense?

(Image: Shutterstock)

PostRank Analytics: Missing Link Between Social Media Engagement And Web Analytics

I love Google Analytics. Google’s free tool offers easy-to-use analytics perfect for small or mid-sized businesses, is easy to install and, perhaps most importantly, is free. Unfortunately, in the world of social media, analytics focused on your own site can only tell you so much. They leave a gap and, for companies involved in online discussions, it’s an important one.

Today we have a new service to help fill that gap.

Introducing PostRank Analytics

PostRank Analytics, launched today, takes top-level data from Google Analytics and layers social media engagement on top of it.

I’ve had a chance to test the service over the last little while. I’m happy to say it has a lot of potential for personal and corporate bloggers alike, at a very low price point.

Overview

The overview page for PostRank Analytics shows quick at-a-glance metrics, including:

  • Page views
  • PostRank’s engagement score
  • Twitter followers

You can also see trends for the first two over a period of up to three months. Blog posts are also featured on the appropriate days.

Mousing over a particular day reveals the exact numbers for that day, while clicking on a blog post pulls up deeper measurements for that post.

Analyze

Digging down into the analysis section of PostRank Analytics lets you access more detailed metrics on each of your blog posts.

An initial screen lists posts in reverse chronological order, while clicking any post mines right down to show such measures as average time on site, engagement on each social media platform (such as Twitter, FriendFeed, Tumblr, etc), and bounce rate.

The page also gives a complete history of conversation about your post on those third-party services. One particularly useful aspect of this feature is that it attempts to make it easy to reach people talking about your content by identifying their presences on other sites.

Your own concierge

Another useful feature of PostRank Analytics is the option to have daily reports delivered right to your inbox with a summary of the previous day’s activity.

The concierge report is a stripped-down snapshot of activity, showing total page views and engagement on your site along with activity on posts such as views and additional conversation over the day. While you may not find it useful if you’re highly involved with your site, it may be a useful tool for people who aren’t able to pay close attention to goings-on.

Key Points

I like PostRank Analytics for what it provides now, but I’m also excited about the potential for new features. Right now, the level of data pulled in from Google Analytics is relatively small, but there’s room to build on this as the service goes through iterations. I’d be interested, for example, to see which posts led to the most conversions and to track that against engagement.

The service is most likely to be attractive to people with well-established sites or those working on corporate sites. The price of $9 per month is low enough to make the service very accessible to beginners, however I think they are less likely to want to pay for analytics at an early stage.

I really like the inclusion of commenters’ other social media profiles in the service. The addition of ready-to-hand research on commenters is useful for people trying to decide whether to respond to individual conversations.

I’m really happy to see PostRank roll out a consumer-focused service that they can monetize. An analytics service was a logical direction given the wealth of data they have on engagement, and in my view is a useful addition to their portfolio.

Conclusion

PostRank Analytics provides the missing link between social media engagement and web analytics. The service is useful as-is, and has substantial potential for expansion.

At this price point, PostRank Analytics is one to explore now, and to watch for the future too.

PostRank Analytics

PostRank Analytics - Detail

Enough With Misusing Social Media ROI, Already

ROI-graph

I’m a little tired of abusing the term “ROI” – giving it new meanings just so they can say they’re measuring it. “Return on Interaction”… “Return on Engagement”… enough already.

Breaking news: ROI may well not matter for your social media program. (Edit: At least, not as a direct, immediate metric.)

Except this isn’t breaking news – people just don’t seem to hear it.

Here’s a definition of ROI from Wikipedia:

“Return on investment (ROI) [...] is the ratio of money gained or lost (whether realized or unrealized) on an investment relative to the amount of money invested.”

There’s even a formula:

ROI

ROI is a finanical term. It has a set definition, which carries plenty of weight in companies. However, that doesn’t mean you can always relate your programs directly to it.

For the formula to work, you need to know the cost and benefits of your program in dollar amounts. You should know the cost of your investment, but the gain may be hard to attribute (especially to a single factor). What’s the gain from improved customer service? From relationship-building? From increased employee engagement?

Sometimes you CAN identify a specific gain from your investment. Sometimes you can tie specific activity to conversions and have a specific value for those conversions. In those cases, you’re in luck – you’ve hit the communicator’s nirvana. The rest of the time, just accept it:

ROI may not be the right measurement for you.

Does that mean your program isn’t valuable? Does that mean you’ll never get executive sign-off? Does that mean it’s not worth measuring your program?

No.

It means you find appropriate ways to tie measurement back to your objectives. Those last four words are key: “back to your objectives.” Because everything should lead back to them.

As we’ve navigated through this recession, we’ve seen clients become (rightly) more and more focused on measuring outcomes, not outputs. It’s music to my ears, because this gives us the opportunity to (a) measure the heck out of a program and (b) adjust programs to ensure they achieve the right results for the client.

Those measurements don’t have to lead to a financial formula; they just have to tie back to your client’s goals. Do they want to drive sales? Address customer issues? Be perceived as leaders in their market? I could go on and on. Each of these has different end metrics, along with different proxies along the way. They’re all valuable.

So, please – enough with “return on influence” and other variations on the term “ROI.”

The fact that you’re not measuring ROI doesn’t mean you’re not measuring success or impact. In fact, it may just mean you’re measuring the right thing.

What do you think?

Update: Oliver Blanchard made an excellent point that I neglected to include here – Ultimately, all of these measures SHOULD feed back to ROI. If your company isn’t tying its activities back to that eventually, you risk both the cost of an ineffective program and the opportunity cost of missing more effective investment elsewhere. I would add that there may be intermediate steps between your program and the ROI calculation. Making-up new metrics because you can’t tie directly to ROI does nothing to help you.

(Images: Investopedia, Shutterstock)

A Little Shortening Goes A Long Way

As micro-blogging services like Twitter, Brightkite etc grow, URL services are becoming a central part of every social media junkie’s toolkit.

However, as Danny Sullivan pointed out recently, not all URL shorteners are created equal. Your URL shortening service can detract from your online marketing, or it can enhance it.

If you use the TinyURL service to which services like Twitter default, you’re missing out on a raft of useful information that can help to demonstrate the value of your outreach.If you’re using these services for your own personal uses you may not care; if you’re doing it as part of your job then you should.

Services like bit.ly give you detailed analytics, including:

  • Live click-through count
  • Breakdown of the locations of visitors
  • Breakdown of the services people are using to access the shortened URL
  • Social media conversations featuring the link

Of course, this isn’t going to replace other forms of measurement, but it can provide specific data on your individual activities. In an economy where measurement can give you the edge you need to justify your budget, can you afford to ignore this extra level of measurement?

Test-Driving The New Radian6 Features

Radian6 really got its groove on this weekend, with a whole raft of new changes that close the gap with its competition in some areas and set it ahead in others. I’d had a heads-up that some of them were on the way, but this weekend was the first time I’d had a chance to play around with them. 

 I use Radian6 pretty much daily and have fed a number of thoughts back to the team there, so I was excited to see the announcement of this latest round of changes. Here are the major changes from this release, along with my take on them.

Real-time notifications

One of my big peeves with Radian6 has always been that you could only get notifications of new search results once per day. In a constantly-evolving online environment, once per day simply doesn’t cut it.

With the new enhancements, this issue is removed; replaced by near real-time emails and/or IMs – available through the “configuration” tab. The IM function isn’t compatible with Live Messenger so isn’t much use to me unfortunately (we use MSN at work), but is a nice addition.

This new functionality is only available for new topic alerts, which may be a little irritating, but it’s more than worth taking the time to re-create your alerts so you can be notified more frequently about new posts.

For an example of how effective the new alerts are, about an hour after posting the images from this post to Flickr I received a message from Marcel Lebrun, CEO of Radian6, whose alerts had picked-up my images and who smelled a blog post in the works. Marcel also pointed out another neat feature of these “as-it-happens” alerts:

“When you click on a tweet via As-it-happens alerts, you get to see [a] special page with tweet, bio, recent tweets to/from and can engage.”

I’d missed this until that point – you can now log-in to your Twitter account from the conversation sidebar (see below) and engage with people from there.

“Conversation sidebar”

A little while back Radian6 added workflow features to their service – the ability to assign posts to others, allocate a status and classify posts, and add notes and tags. Very useful, but a little clunky.

With the introduction of the conversation sidebar, Radian6 has taken its first steps towards a workflow system that I can see us using.

The conversation sidebar appears on the left of your browser when you click a link in a notification email, an IM message or your “river of news” widget, while the post you’ve selected appears on the right. From this sidebar you can assign posts to other members of your team, identify your engagement with the post and any future action, assign sentiment and classify the conversation (leads, complaints, compliments, etc).

The “Add to the Conversation” field is a little misleading – the field simply adds a ‘note’ to the post in Radian6. With a title like that, I would expect it to post a comment to the post in question.

This is a significant addition to the Radian6 workflow which, despite its clear usefulness for groups, has been underused so far in my view. The next piece to this particular puzzle is real-time emails when you assign a post to someone, so they don’t have to continually log-in to Radian6 to find their newly-assigned posts. 

As the announcement to customers stated, “The Conversation Sidebar will help enterprise teams scale their engagement, coordinate their community outreach, and track and analyze their external conversations.”

Source Tags

You can now tag not only posts, but also sources. I can see this being useful for categorizing sources, flagging that you’ve previously responded to them, noting where they’re from etc.

The source tagging option is available from the workflow in the “river of news,” and from the conversation sidebar (you can see it in the image above). However, this option does not seem to be available for forums, which seems odd. Not sure why that is – while you might not want to tag things as specifically, it would still be useful.

Comments

This is a big deal for me: the addition of comments to Radian6′s coverage. This was a big gap between Radian6 and its competitors previously, and a big time suck for those of us monitoring online. Pulling blog comments into Radian6, while still allowing the option to exclude them from volume analyses, is very powerful.

Why does it matter so much? As I outlined in my PodCamp Toronto presentation recently, let’s say a car enthusiast writes about GM‘s latest car. If you were Scott Monty at Ford (note: this is hypothetical – I have no idea whether Ford uses Radian6 or not), that would likely not show up in your search results. If, however,  the comments took a swerve and the conversation focus switched to Ford’s latest offering, that still wouldn’t have shown up… until now, anyway. Now that comments are indexed, the comments referencing Ford would now show up in Scott’s dashboard, and he could decide whether or not to engage.

Interestingly enough, the comment indexing is provided by BackType, about which I’ve written in the past. Christopher Golda of BackType was actually in the crowd during the presentation I mentioned above.

One important point: if your profile is close to the boundary between different pricing levels, note that the addition of comments will drive up your montly search volumes and have a knock-on effect on pricing.

New metrics

The new rollout gives Radian6 users a couple of new metrics to use when looking at influence.

While users have always been able to see the number of “on-topic” inbound links (though how they decide what is “on topic” is beyond me) to posts in their search results. The new release adds the total number of inbound links (according to Google) to the analysis widgets. Very handy, and very easy to spot.

The other new metric is perhaps more useful as it’s the first thing, beyond forum views, to track traffic numbers in Radian6. The new release adds Compete.com website traffic statistics into the influencer widget.

However, there are a couple of “buts” here. The first is that this data isn’t free – it runs to $50 per month, per topic profile. The other “but” is that, as with many services, Compete is great but for sites with smaller traffic volumes (like mine), you get “rough estimates.” It also focuses on US visitors. Still, Compete is the leader in this kind of analysis, so this is another step in the right direction.

Content segmentation and analysis

You can now segment your analytics even more effectively, with break-downs available on:

  • Language
  • Region
  • Media type
  • Engagement level
  • Source tag
  • Post tag 

The biggest develop in this segmentation for me, though, is the ability to segment by sentiment.

While, in the past, you could allocate sentiment to posts, until now you couldn’t graph it so it was essentially useless. I’m a little disappointed to only see positive/neutral/negative as options and not the nuances (the “somewhat positive” and “somewhat negative” posts are grouped as “other”) but, again, it’s a good step in the right direction.

You can also sort your analysis widgets by numerous metrics:

  •     Number of posts
  •     Comment count
  •     View count
  •     Vote count
  •     Twitter followers
  •     On-topic inbound links
  •     Total inbound links
  •     Number of unique sources

As the announcement notes, “Want to know which keywords or topics generate the most commenting activity? Which blog post generated the most Twitter impressions? Now you can see the buzz around your topics at a glance.”

Conclusion

This is an excellent set of new features for Radian6. I have quibbles with a few things here and there, and the workflow in particular is a work in progress, but the product is ever-evolving and this is a strong release that adds significantly to Radian6′s usefulness.

The most important features, in order of importance (from my perspective):

  • The addition of comments;
  • Real-time alerts;
  • Graphing sentiment;
  • Workflow improvement via the conversation sidebar (would be higher with the addition of email alerts).

PodCamp Toronto 2009 – Bigger, Better?

I’m at the end of an exhausting but wonderful weekend. PodCamp Toronto 2009 was held over the last two days and, as an organizer, it was an immensely rewarding experience.

I’ll have posts on various topics from this year’s event throughout the week, so I’ll keep this post brief. 

First, a few interesting points about PodCamp Toronto this year:

  • More than double the size of PodCamp Toronto 2008 – between 500 and 600 people this year (over 500 confirmed)
  • Top trending topic on Twitter on both days; pretty much all day Saturday (see below)
  • Hundreds of photos posted on Flickr already
  • For a short time, PodCamp Toronto was “bigger than Jesus” (hat tip to Bob Goyetche and Mark Blevis) at one point this weekend

My Presentations

I gave two presentations this weekend. One was planned in advance – thinking about and doing social media measurement takes up a good chunk of my time nowadays, so it made sense to talk about it and I signed-up to present on that several months ago.

My second session was a little more impromptu – I woke up on Sunday morning and decided I felt like presenting again, so I signed-up to host a session on the ethics of social media PR. Happily, both sessions were well-attended and well-received.  The slides for each are embedded below.

I’ll have more thoughts, and a couple of interviews, from PodCamp Toronto over the next few days. 

For now, if you went to PodCamp this weekend, what did you think? What was good/bad/indifferent?

3 Steps To Better Objectives

What's your target? “Increase sales” isn’t a good objective.

Neither is “increase web traffic,” or “increase awareness,” or “more customers.”

Why?

Because you have no way of measuring success. If you can’t measure success, then what use is your goal?

I have to bite back a visceral reaction whenever I see vague goals in a communications plan. They’re toothless, they’re meaningless and they turn what could be a selling point for us (compelling objectives) into a waste of space. Sure, they provide a vague focus for work, but there’s no spine to them.

Let’s say your initiative – your communications; your ad campaign; your promotion – resulted in one additional customer. Is that success? Maybe if you’re Boeing or Bombardier, where one additional customer means multi-million dollar deals. If you’re McDonalds or Lays then perhaps not.

While the the kumbaya/let’s-all-get-along discussion inside the blogosphere might find that kind of objective acceptable, if you’re competing in real life with marketing/advertising agencies and other corporate departments for limited dollars, you need to be more specific and you need to talk outcomes, not outputs.

Creating better objectives

A credible goal needs to have three components:

  1. Change – What will you improve?
  2. Quantifier – How much will it improve?
  3. Deadline – When will you do it by?

A call to action

Corporate folks

If your agency walks in and says their goal is to increase your sales for next year, ask them by how much and by when.

If they say they’re going to improve your reputation online, ask them how they plan to measure that.

Let’s face it, times are tough. You need to know that you’re spending your dollars in the right areas.

Hold your agency to account.

Agency folks

Pre-empt this discussion. Walk into the room with your goals fleshed-out.

As anyone in PR knows, the end-goal effects can be hard to quantify so don’t shoot yourself in the foot and aspire to something you’ll end up not being able to prove. Use proxies.

You may not be able to directly prove sales, but you can certainly find a way to draw a line between things you can affect and the big-picture end-goal.

For example, instead of “improve your online reputation,” try something like:

Goal: Improve [brand X]‘s online reputation by:

  • Increasing the proportion of positive online comments about the company, compared to negative and neutral comments, by 10 per cent over the next six months;
  • Increasing the volume of mentions of [your brand] online by 15 per cent by March 2010;

Yes, external influences occur. Yes, they’re unpredictable. Just be ready to discuss those when you review your program after the deadline. Don’t let them prevent you from setting useful objectives at the outset.

What do you think?

How To Write A Good Communications Plan – Part 13 – Evaluation

Measuring This is it – the last stage of preparing your communications plan – evaluation.

As with several parts of this communications planning series, the stage at which you write this part of your plan is fairly arbitrary. I recommend you turn your mind to it after, not before, you finish considering your analysis, objectives, strategy and tactics (you do need to know what you’re measuring, after all), but beyond that point it’s largely up to you.

Evaluation is a tough area to tackle, and one that’s often neglected in public relations. There are plenty reasons for this:

  • The challenge of trying to find a measurement system that accounts for the wide variety of tactics possible in a public relations campaign
  • The reluctance of clients, be they internal or external, to dedicate budget to evaluation
  • The lack of well-established criteria for measuring social media success
  • The fast-moving pace of communications that moves us on to the next announcement as soon as the last one is finished.

Your goal for this section

Your goal in your evaluation section is to lay out how you will measure your communications success. In a high-profile initiative this may be through the various stages of your announcement (we identified three – pre-announcement, announcement and post-announcement, when we looked at tactics earlier); in others, it may have a smaller scope.

Staged Measurement

If you’re planning a staged rollout of your communications program, try to measure your results over time. Alongside providing more credible results, this has the added benefit of allowing you to take corrective action if you sense your activities aren’t getting the desired results. Take a look at the different milestones you’ve identified for the project and consider which are suitable points to measure at.

Of course, you should also measure at the end of the initiative to see whether you’ve accomplished your objectives. Ideally, you’ll be able to compare that to the results showing whether the business objectives were accomplished too.

Potential Metrics

I’m certainly not an expert in measurement tactics, but here are a few measurements you may want to consider, depending on your objectives:

  • Media coverage
    • How much coverage did you receive?
    • What was the tone of that coverage (positive/negative)?
    • Which media outlets was the coverage in? Where in those outlets? What’s the audience of those placements?
    • Did you achieve the desired visuals?
    • Did they pick up your key messages?
    • Were your spokespeople quoted?
    • Were the mentions of your initiative the focus of the coverage, or a side note?
    • Methods for achieving these metrics vary. While I haven’t used it personally, the Media Relations Rating Points system has achieved some traction (see Ben Boudreau’s One Degree post for a case study).
  • Interactive
    • How many visitors saw your content?
    • How long did they spend on the site?
    • What pages did they visit?
    • Did they hit specific landing pages?
    • What was their bounce rate?
    • What was their conversion rate (identify a goal for visitors – purchase/registration/download, etc.)?
    • Social media measurement is even more debatable than regular PR. Comments, inbound links, etc are lovely, but at best they’re just proxies for more meaningful measurements.
  • Stakeholders
    • How did your stakeholders react?
  • Public inquiries
    • How many letters/emails/calls did you receive on this topic? Is that higher or lower than usual?
    • What was the tone of the incoming correspondence?
    • What did the correspondents say/ask?
  • Benchmarking
    • Conduct market research/polling before and after (perhaps also during) your communications to show improvement in metrics over time, for example in public attitudes
    • Focus groups

These are just a few metrics. What others can you suggest?

The “Communications Plan” Series

This is the final post in my series of 13 posts on DaveFleet.com exploring how to create a good strategic communications plan. To read the rest of the series, check out the other posts here.

(Photo credit: verzerk)