Archive for the ‘measurement’ Category

From One to a Million: Managing Social Media at Scale

What if you had to re-examine your assumptions around social media? What if, instead of thinking about conversations in ones and twos, you had to think about them in thousands and tens of thousands? What if you had to manage dozens or hundreds of properties, and millions of fans? What would change?

Last weekend I presented a session at PodCamp Toronto entitled “From One to a Million: Managing Social Media at Scale.” The goal of the session was to prompt people to question some of the norms espoused by many ‘experts’, who have never had to manage social media programs at anything beyond a small scale. Norms such as the idea that you “need” to talk to every person who engages with you – something that is feasible at small scale, but infeasible when you get into the tens of thousands of replies weekly.

This is not to say that those norms are completely incorrect, but that there is a practical reality for brands operating at scale – structure changes, processes change and the norms have to change.

Key points from the presentation:

1. Structure: How do you structure to handle social media at scale?

Brands need to grapple with structural decisions at a global scale:

  • How much do you centralize vs decentralize control?
  • Do you house social within the corporate HQ or at the business unit level?
  • Do you aim for consistency and economies of scale or responsiveness at a local market level?
  • Do you impose social media on the enterprise or allow it to grow organically?

There’s no right or wrong answer; the decision depends on objectives, on your broader business structure, on the scale of your social media activities, on your business’ culture and on the resources you have to hand, among other things.

2. Community Management: How do you go from 1:1 to 1:1,000,000?

Community management at scale requires brands to reassess the norms they hear espoused daily. I offered seven pointers for scaling community management practices:

  1. Moderate to deal with trolls (with an affectionate prod in the slide at Scott Stratten) – if you operate a social media program at scale without moderation, you’ll spend your life dealing with trolls and spammers
  2. Embrace proactiveness – don’t wait for people to come to you; use analytics and insights to drive proactive content to answer questions ahead of time
  3. Recognize you can’t talk to everyone – at some point you need to prioritize or you will drown
  4. Respond publicly when possible (and when appropriate) – answering publicly lets other people (a) see you being responsive and (b) see your answers and possibly answer their own questions
  5. Help customers to help customers – successful companies in the social support space leverage customer forums to help customers answer each others’ questions, and step in when questions go unanswered at first
  6. Build an army of advocates – educate, empower and reward your biggest fans for engaging for you
  7. Know your customer – know who they are, what they want and how they want it to serve information most appropriately for them

(Check out my related post on tips for scaling customer service)

3. Content Strategy: How do you stay engaging while driving business results at scale?

Content strategy is a shiny object right now (in a stroke of amazing timing, Edelman appointed Steve Rubel to the new post of Chief Content Strategist yesterday – congrats Steve). I offered three broad categories of ways to resist the myriad pressures that face social media teams within corporations, and to stay on strategy:

  1. Know your objectives, and use them as a decision making framework.
  2. Know your channels, your audiences and the difference between them.
  3. Execute with rigor and optimize relentlessly.

4. Measurement: Turning a challenge to a competitive advantage

Measurement has historically been a pain point for many PR practitioners, but it’s a point of passion for me – I truly believe that effective measurement can be a differentiator for companies’ social media programs. When you begin to activate social at large scale, statistical analysis of content and program performance can yield invaluable insights.

I’ve in the past on ways companies can improve their social media measurement; this time around I offered another five tips:

  1. Focus on the right things – measure the right things for the right audience to meet their objectives.
  2. Connect your metrics with your objectives – don’t measure share of voice if you’re looking to improve the responsiveness of your customer support, for example.
  3. Know what the numbers mean – do your research and don’t let companies lead you down the garden path with made-up numbers and meaningless multipliers.
  4. Generate and drive insights throughout your program – look at your foundational always-on activities (your program is always-on, right?), at point-in-time campaigns and at the broader conversation ecosystem for insights.
  5. Use full-program measurement – set measurable objectives, use insights from past programs to fuel program development, course-correct throughout and measure results to drive insights for future work.

This was the first time I had presented this deck. I would have loved to have another 15 minutes longer to incorporate more practical pointers, but this provides a solid high-level overview of how to leverage these four elements of a program both at scale and more broadly. I’d love to know what you think, though – let me know in the comments below.

Six essential shifts in social media strategy

We’ve reached a critical point in the evolution of social media as a business tool. Gone are the days when the GMOOT (Get Me One Of Those) approach will get you anywhere – simply having a Twitter account, or a Facebook Page, isn’t enough. We’re at the point of social media saturation, and something’s got to give.

So began the session description for my recent presentation at BlogWorld New York. The crux: that the days of social media as an experiment are over – it’s time for a more mature approach to social media within companies in order for social media to be viewed as a sustainable communications and business function.

Unfortunately, we’re also at a point where pursuit of the shiny object has reached an extreme, and that this pursuit is conducted within an increasingly transparent fishbowl while armchair critics circle, waiting for the next “fail” from companies.

In this environment, where transparency and scrutiny are paired up with a shift in focus from experimentation to results, and yet where the allure of “the next big thing” persists, companies need to structure and approach social media differently.

My presentation focused on six essential shifts that I see in how many businesses approach social media strategy. Of course, not all companies are in the same situation. Some with mature programs have evolved beyond this stage; some face just a few of these shifts; others face them all:

  1. Moving away from shiny objects and towards social business
    1. Asking “why” to understand demands
    2. Building a social media infrastructure to support the social brand
    3. Taking baby steps in implementation – from crawl, to walk, to run, to fly
  2. Setting better objectives for social media
    1. Setting SMART objectives
    2. Tying back to broader business goals
    3. Staying clear of the “how” and “what” when setting objectives
  3. Measuring effectively against those objectives
    1. Focusing on the right numbers for the audience
    2. Understanding what numbers really mean
    3. Avoiding made-up numbers
    4. Measuring to drive insights alongside determining results
  4. Breaking down silos and integrating across functions
    1. Approaching social media as an integrated function
    2. Breaking-down silos through day-to-day tactics
    3. Integrating through reporting structures, governance and social media organizational models
  5. Planning and executing content more strategically
    1. Considering content objectives
    2. Identifying appropriate content sources
    3. Fine-tuning execution via appropriate content volume, mix and format
  6. Engaging effectively to build relationships and communities of interest
    1. Embracing negative and neutral conversations
    2. Establishing processes to minimize risk

How about you – have you seen companies needing to make these improvements to their social media strategy?

For more on the topic, check out this excellent write-up of my presentation over at SmartBlog for Social Media.

Thanks once again to Rick, Dave, Deb, Shane and the rest of the BlogWorld team for the invitation to speak. This was my fifth BlogWorld presentation, and I always enjoy it. 

Six important shifts for social media in 2012

It’s hard to believe we’re about to tick over to another calendar year. So, as usual, I got to thinking about the shifts I think companies need to make in their social media activities in the next year.

These aren’t necessarily trends that are already happening (although I’d like to say they are), but they’re certainly where my head is at and hopefully where others are, too.

Here are six shifts I hope to see in social media use by business in 2012.

Better objective-setting

Over the last couple of years, we’ve seen a slow maturation in the way companies develop their objectives for social media. My hope is that this will continue in 2012. That means fewer companies treating fan or follower growth or video views as goals, fewer made-up numbers and more focusing on business outcomes – sales, cost savings, customer/employee retention etc.

More effective measurement

As companies get better at setting objectives for social media, they’re going to need to get better at measuring against those new objectives. That means shifting focus away from  anecdotal evidence and simple outputs, and looking at indicators of the behaviour you’re looking to drive. It also means taking a closer look at the reporting of that measurement. See my recent post on five ways to improve your social media measurement for more on this.

This will be accompanied by increased realism over social media results. I’m currently reading a book that points to a multi-national company having 27,000 Twitter followers as an indication of social media success. Let’s face it, that’s unlikely to move the needle for lots of companies. As companies focus-in on reporting business objectives, we’ll see a continued shift away from high-fives over anecdotes and minor wins and a more hard-nosed focus on what really matters.

Improved Integration

Key to measuring more effectively, but with far, far broader effects, integration (and the breaking down of silos) will become even more key in 2012. The smart organizations have already figured out that social media works best when supported, and supporting, other forms of communications; look for more companies to mandate a silo-busting approach over the next year.

Strategic content planning

As organizations increasingly adopt the role of media companies in their online communications, watch for content strategy to receive greater focus in 2012. That means shifting from a “we have to fill these content slots” approach to one that carefully considers the objectives of each piece of proactive content and why it deserves its place in the content calendar. Sometimes it might be to drive community engagement; other times it might be to drive business conversion, and so on.

Increased search focus

An increased (and improved) search focus sits alongside more strategic planning of content. It means broadening the scope of how you target content, from point-in-time to point-in-lifecycle – thinking about what people are looking for at their stage in whatever process you’re targeting, and helping them through that and on to the next stage. That could be a stage of the purchase cycle, it could be a stage of the support process, or any number of others that you choose to focus on (thinking back to objectives).

Focusing on the less-shiny object

This is a big bucket of all sorts of increases, but my hope is that as companies move away from shiny-object snydrome in 2012 they start to take a more sophisticated approach to the less-shiny objects – policies, processes, listening, crisis plans etc – or, more formally put, to social business.

Social business

For me, this is an exciting time. I’m jazzed to see more mature use of social media help it to evolve into a more powerful tool for organizations – “life after likes“, as David Armano puts it. This is the cool stuff – the stuff that will move the needle and add real value for companies.

That makes the non-shiny objects the shiny ones for me.

 

Five ways to improve your social media measurement

We’re past the point  where a “get me one of those” approach to the latest social shiny object is a viable approach. As business use of social media continues to slowly mature, measurement is becoming more and more important to justify the investment in social activities.

Last week I spoke on a roundtable on Practical Social Media Goal Setting, Measurement and ROI, along with Janet Fouts, Steve Farnsworth and Brian Rice. One of the most interesting questions asked related to the mistakes that organizations make around measuring social media. Rather than focus on those mistakes, here are five ways to avoid them – five ways to improve your social media measurement and reporting.

1. Focus on outcomes over outputs

The number of Tweets you post, or replies you generate, may be interesting, but what value do they drive? Instead, focus on the outcomes of those activities – how many leads did you generate? How much money did you save? How many event registrations did you drive?

It can sometimes be hard to accomplish this – especially from the agency side – as you often need to work with many business functions to determine this (sales, IT, marketing etc). The more you can push in this direction, though, the better off you’ll be.

2. Set measurable objectives

Measurement factors into SMART goals in several ways.

Firstly, considering insights from previous activities (previous reporting periods or previous campaigns) to set appropriate expectations for the time-period set.

Secondly, ensuring that the objectives that are set are measurable so you can measure the ultimate success or failure of your program at the end.

3. Determine the purpose of your report

I’ve seen way too many reports that lay out the objectives of a program, then report on measurements that have nothing to do with those objectives.

If you’re looking to get ten thousand sign-ups to a new service, why would you report on the number of retweets of your content? Or the sentiment of coverage? Those numbers have value when you’re looking for ways to optimize your activities, but when it comes to measuring against your objectives, the connection there is weak at best.

Before you begin working on a measurement report, determine what the purpose of that report is. Is it to optimize content? Provide insights to fuel products, services, messaging, etc? Or to determine the success of a program? Determine the purpose of your measurement and tailor your approach to it.

The audience of your report is often tied closely to its purpose. Who are you writing for? The answer to that question – whether it’s community managers or senior executives – should determine the kind of insights you provide.

Check out Jeremiah Owyang’s great post on the social media ROI  pyramid for a great way of thinking about this.

4. Focus on driving action

Number soup doesn’t help anyone. To make your reporting valuable, make sure you seek to drive insights and action with what you report. To quote Rob Clark, look not just at the “what”, but the “so what” and the “now what”.

5. Measure before, during and after

The measurement process doesn’t start at the end of a project; it starts at the beginning, with setting your objectives, and continues throughout the work. Measurement, done right, fuels your objectives, sets a baseline against which you can measure, enables course adjustments during the activities and measures against your objectives at the end.

Doing these things won’t make measuring your activities any easier, but it will make your measurement and reporting more effective, and will help you to improve your social media activities over the long run. That may not be shiny and glamorous, but it’s effective and valuable.

(Image credit: Jeremiah Owyang)

Return On Influence Can Return From Whence It Came

The Harvard Business Review recently published a post entitled “Return on Influence, the New ROI“. In it, the author suggested that marketers consider the use of “Return on Influence” as a metric for measuring social media activity.

What is this metric, you ask? To quote the post:

“Divide the total revenue generated via social efforts by the number of social media fans and followers, and you get a per-fan/follower value.”

There you have it – your “new” ROI – return on influence.
Really? Looks to me like that’s “Revenue Per Fan/Follower”.

Sorry, but this kind of black-hat math just doesn’t cut it. There are so many holes in the post, it’s hard to know where to begin (fortunately, Olivier Blanchard and Katie Paine did, in the comments). Still, I’ll take a stab, because I think it’s important that you, me and everyone in this space stop using BS metrics to justify social media activities and start to tie them back to business objectives.

I’ve written on this before (check out this post from two years ago) but here we go again…

1. Measuring a return requires that you compare outcomes to the input

How can you calculate a return on something without knowing what you put into it? My head hurts. This isn’t a true “return” metric; this is a poor attempt to calculate the value of a fan (without considering many of the factors in play even in that instance).

2. ROI is ROI, not Return on Imaginary Numbers

ROI has a formula. It goes like this:

(Gain from investment – cost of investment) / cost of investment

This isn’t up for negotiation. It’s a business staple. Please – if you value your job – don’t walk into a boardroom and try to sell your CFO on your fan numbers. Don’t try to sell them on retweets, or replies, or anything like that (they’re useful, but not in that context or for that audience). Show them the return that you’re able to generate for the business.

It might be hard to tie social media activities directly back to ROI, as there’s rarely a direct, solid line to be drawn (it’s extremely hard to say what, beyond the final trigger, influenced a decision to purchase, for example, but it doesn’t mean those things weren’t worthwhile). However, solid business objectives do tie back.

Which leads me to the next point…

3. Return on Influence has nothing to do with business objectives

This is something I’ve been putting a lot of thought into recently – ensuring that the social media activities we plan tie back to business objectives for our clients. Sometimes that’s sales. Sometimes that’s reduced customer churn. Sometimes it’s lowered costs.

It’s never “increasing the revenue per follower” or the “return on influence”.

 4. Measurement should be activity-specific

Imagine going pitching a metric like “return on PR”. The conversation might go something like:

You: “We calculate Return on PR by looking at the revenue generated from PR against the volume of releases we put out…”

Boss: “Get out.”

This idea is similarly ridiculous. Measure an activity, not a medium. You want to measure the ROI of a tweet? Fine. Figure out what it cost to draft/approve/publish it (time is money) and how much revenue it generated (assuming it was sales-focused). There you go – you can calculate the ROI of the tweet, and you haven’t broken a sweat yet.

Don’t try to measure the ROI of social media, or of “influence”… please.

5. Followers and fans don’t define influence

Every time someone uses reach metrics to try to define influence, a great hue and cry goes up. “It’s not reach, it’s context!” they cry. It’s true. Plus it’s a bunch of other things.

Folks like the team at Traackr have realized this, as have those at PeerIndex. Klout has cottoned-on, too, with its topic pages (although I’d still like to see them go much further down that road).

If you measure your results based on fans and followers, don’t expect senior leadership to buy into your plans for long.

6. Please – PLEASE – stop creating fake numbers

Like Ad Value Equivalency (AVE), this number tries to force a square peg through a round hole.

AVE aimed to show the value of media coverage if that same coverage had been a paid ad rather than earned media. It was bullshit, plain and simple, as it didn’t account for sentiment, credibility or any other measurement that fit around it.

There are plenty of other metrics thrown around that apply arbitrary (and, often, opaque) formulae to generate meaningless values for social media activities. I can’t stand them (plus, they violate the Barcelona Declaration of Measurement Principles, which the world’s biggest PR firms (ours included) have endorsed.

Please – let’s stop creating fake numbers and take a long, hard look instead at how we can tie our activities back to business objectives, and measure against that.

Fair?

Compete Responds To Criticism

Last week I voiced some concerns around the traffic numbers that Compete.com reports for websites. My primary concern was that, after seeing nearly a 90% variance between their numbers and those from Google Analytics on my own site, I couldn’t trust that their numbers for other sites would be correct. This is important, as we often use Compete.com and similar sites to report on potential reach of coverage for our clients.

I was pleased to see that Compete responded, multiple times — in a comment on my post; via a tweet and then subsequently in email.

Given the interest I saw from other people, I obtained Compete’s permission to publish our exchange.

Comment on my original post:

Hi Dave – Matt here, Client Relations Director at Compete.com.  I am happy to do my best to clear up any confusion if you reach out to me at support@compete.com.  Compete is based on a sample of 2 million, US based panelists.  We have a small sample warning on your website, which means we have limited data on your domain.  Even when considering your GA numbers, you are well down the long tail of the internet.  The code you’ve installed, it is just for audience profiles and it is only found within that tab of the product.  It does not impact any traffic numbers.  In any event, I’d be happy to explain more if you would like to reach out.

While I (and several people on Twitter) wasn’t thrilled with the tone of “you are well down the long tail of the internet” (which sends a message of “just wanted to remind you you’re nothing to us”), I took Matt up on his offer and emailed their support address:

Hi,

I’m reaching out following a tweet from your twitter account regarding a recent blog post of mine (http://t.co/lCXr4UN).

I’d love to understand more about why the numbers seem to diverge so much from what I’m seeing on my website logs.

Thanks,

Dave

Sure enough, a day later I received a reply:

Hi Dave,

Thanks for reaching out to us about your site. I know that it can often be confusing [Dave: no, it's not confusing; it's just irritating. Moving on...] when comparing the traffic numbers for local analytics (google, omniture, etc) to the numbers on Compete.com. At the core the methodologies are like comparing apples to oranges, they’re both fruit – just produced from different trees. Think of compete numbers as an orange – a U.S. based research numbers that help you understand your size and trends against your competition. Local analytics is more like the apple that helps you understand what’s happening on your site so you can improve your visitor’s experience. They’re great supplements to one another in terms of getting a more complete picture of the internet, but are inherently very different in the approach you take to consuming the data sets.

From a more technical perspective panel-based clickstream data (Compete.com) and web analytics data (local analytics (Google Analytics) and server logs) stems from the underlying methodologies that each approach use. At a high-level, panel-based providers like Compete measure online behavior based on consumers, whereas local analytics measure similar behaviors based on cookies. The consumer metrics that panel companies provide are based on statistically-derived estimates that are derived from a representative sample of consumers; in this instance, the behaviors of the sample are weighted and extrapolated to represent the entire internet browser population. The cookie-centric metrics that web analytics companies provide are developed on simple counts of cookies for all of the web pages that are a tagged on a site or a set of sites; when a consumer visits a specific page on a site, that visit is counted by the web analytics platform.

Both approaches have their strengths and limitations. Panel-based measurement provides excellent insight into visitor demographics, what consumers do across all of the websites they visit and analysis over long time periods. The limitation of consumer panels is that they sometimes do not provide sufficient sample to measure “low incidence” behaviors such as visiting very small sites, using rare search terms, or interacting with low-traffic pages on specific websites. Compete’s panel is one of the largest in the industry, this helps us ensure that we can measure and report on more of these infrequent behaviors compared to other panel providers.

If you take a look at information which is collected in a similar fashion below you’ll see either no data or considerably smaller numbers than local analytics.
http://trends.google.com/websites?q=davefleet.com&geo=all&date=all
http://www.quantcast.com/davefleet.com

Cookie-based web solutions are good sources of information on all of the behaviors that occur within a website, and therefore can be used to calculate and optimize site flow, conversion rate and other onsite activities. In this manner, web analytics are not subject to the same sample requirements as panel companies. However, there are some limitations that cookie-based solutions are susceptible to that panel-based measurement services are not. Cookie-based data can be affected and sometimes inflated by the deletion of a user’s cookies, incorrect page tagging, and susceptibility to bots or spiders. Also, the data found on Compete.com comes from our panel of U.S. users, local analytics and server logs collect U.S. and International data.

I hope this explanation can help clear up any questions that you might have with our numbers vs. those seen when looking at Google Analytics and server logs. Please let me know if you have any other questions that I can help you answer. If you’d like to learn more about our methodology please reference our data methodology whitepaper: http://media.compete.com/site_media/upl/img/Compete%20Data%20Methodology.pdf.

Kind Regards,

Lindsey
Compete.com Customer Support

While some people I spoke to felt the answer was a bit condescending, I thought it did a good job of explaining the difference in methodologies between Compete and Google Analytics in plain language. Ok, so now I understood the difference between the two services’ methodologies (I didn’t ask, but it’s helpful nonetheless). The question remained, though – why should people trust Compete to provide data on anything but the top tier of sites on the Internet?

So I asked:

Hi, [I didn't realize until afterward that the original email had been signed -- my bad]

Thanks for the thoughtful response.

I do have two follow-up questions for you:

1. For sites that are, in the words of your Client Relations Director, “well down the long tail of the internet” [Dave: I couldn't resist], do you therefore recommend using other data sources than Compete?
2. Would you mind if I published your response as an update to my blog post? I would love to include your side of the story for people to consider.

Thanks,

Dave

Compete’s response:

Hi Dave,

Thanks again for reaching out, we’re happy to have you publish our dialog and are open to answering your questions about our tools and how it relates to the industry. Though I would request that you include your follow-up question to provide context to our reply.

In terms of data sources, our client relations team advocates for two things. First, a firm understanding of the data you’re looking at through asking targeted questions about the data source. When you’re unfamiliar with a data set this can be really difficult, because you don’t know what you don’t know, right? The second challenge is using the proper tool for the job, compete specifically isn’t meant to replace your local analytics – this is not our goal.

Compete ultimately is looking to provide our customers with a better understanding of their industry and competitors from a research perspective. If your data set on our site is listed under a small sample warning, the data will have more of a directional relevance that gives you an idea of what’s going on for the industry. For sites with more traffic activity, our data is really helpful in understanding the approach competitors take in terms of SEO, SEM, and traffic acquisition to name a few popular insights typically gleaned.

If you take a look at the image attached (Blogging services 2 year category view*) you’ll notice that between the month of March and April there is a slight monthly decrease of about -2.7%. Looking at the trend in 2010 the dip was about -1.1%, so this 2 year view allows you to see that there’s something happening within US online consumer’s behavoir that makes it so that each year between March and April there is a slight dip in the amount of activity to the overall category (which includes domains like blogger and wordpress).

*Compete Categories are groups of domains we organize for our PRO and Enterprise Level Subscribers.

From my personal go-to toolkit, my data sources could include Google Trends – think search data, similar sites, and geo-demographics. If you’re looking for sentiment data visa-vi social networks, I like using Google Realtime to build out timelines that correspond with news and product happenings. I use these services a lot to supplement our information, they’re not a direct replacement in terms of the value add for a lot of digital marketers but they can help give you a better picture of what people were taking about during specific time periods, breaking news, and similar domains that may be of interest to investigate.

When it comes to studying the internet and ultimately the behavoir of users, getting familiar with new tools and services can be a difficult process if you don’t have the resources for full data immersion. If you’re looking to attempt to prompt specific actions you’d like visitors to take, it takes time and practice. One of the things that Avinash* typically preaches, and I like to echo, is that marketers should always be aiming to synthesize what the data trends indicate, rather than simply “reporting”.

Here’s the post* http://www.kaushik.net/avinash/2011/04/difference-web-reporting-web-analysis.html

I hope this helps, please let us know if you have any other questions.

Kind Regards,

Lindsey

A couple of thoughts from my end:

Firstly, thank you to Matt and Lindsay from Compete for their thoughtful responses, and for allowing me to publish this exchange. I appreciate the thought and the time spent on their end.

Secondly, it appears that Compete doesn’t intend for its traffic numbers to be used for analysis – from their response, it appears that “For sites with more traffic activity, our data is really helpful in understanding the approach competitors take in terms of SEO, SEM, and traffic acquisition” insights are the primary uses for their data. Fair enough – I hadn’t thought of Compete in that way before, and it’s good to know that that’s their intent.

Thirdly, I still have two outstanding questions.

Outstanding question #1: How accurate are Compete’s/Quantcast’s/Alexa’s numbers for top-tier websites?

Are we looking at a 5% error margin? 15%? 25%? I’d love to know, because we have a duty to clients to know how accurate the numbers we’re using are.

Compete’s team, for all heir helpfulness, still hasn’t explained why people should trust their traffic numbers for sites (although, frankly, I could have been clearer as to why I was asking). This is still a critical issue – to quote my original post:

Should I believe that CNN.com’s traffic went up by 27% in March compared to February? Should I believe that Mashable’s traffic went down by nearly 30% in the last year?

Why is it important? Because I and many other people look at Compete’s numbers to determine sites’ traffic numbers when reporting on the results of our activities. If we can’t believe those numbers, we need to look elsewhere.

Outstanding question #2: What is the best site – free or paid – for providing reach analysis of lower-tier websites?

I readily acknowledge that Compete is a free service (it doesn’t sound like their Pro service adds much in terms of accuracy – just longer time periods and additional data for analysis), and that perhaps I shouldn’t expect too much from a free service.

I’ll be clear, though: I would be happy to consider paid services if they’re able to offer accurate reports.

Let’s face it – few companies are able to conduct outreach targeting only top-tier websites. Especially when you get into niches, there are relatively few relevant sites with traffic comparable to the top-tier of the Internet. So, where do we go for analysis of the rest?

What do you think? My questions again:

  1. How accurate are Compete’s/Quantcast’s/Alexa’s numbers for top-tier websites?
  2. What, in your opinion, is the best site – free or paid – for providing reach analysis of lower-tier websites?

Is Share of Voice a Useless PR Metric?

This is a guest post by my Edelman colleague Rob Clark

Sometimes you say a word too many times in a row and the word slowly begins to lose meaning for yourself. It becomes foreign gibberish and you begin to wonder if you’re pronouncing this thing correctly or if it was ever really a word at all. Which is all to say that I’ve been giving a lot of thought lately to share of voice (SOV) and I may have passed the threshold where it ceases to hold meaning.

Why do we measure?

We measure because there is a decision we have to make and we are lacking the data needed to take action. So I would like to ask what information does share of voice provide the PR practitioner that guides an action?

In marketing – where all is a funnel down from eyeballs to wallets and the space and time is finite – SOV provides insight into whether your message is drowned by the competition’s. But the real strategic advantage comes in that the cost of the ad space is a known quantity. Knowing what your competitor’s SOV in a market is, let’s you know what kind of resources they are pushing forth. You know which of their products is getting the thrust and in what markets. It shows you some of the cards they have on the table.

But in PR we don’t buy coverage by the pound. We can’t translate ink on the page (or pixels on the screen) into dollars spent on PR. So that set of data is lost to us from a SOV measure.

Editorial – though not infinite – is open to expand and contract. Your amount of coverage can remain consistent but your share contract tremendously as a flurry of write ups about your competitor come out. Let’s say that our client is Widget co. (makers of fine hypothetical examples since 1912). Widget co has a 20% SOV and their nearest competitor has 30%. The following month Widget co is at 18% and their rival at 37%. What decision will this info drive? What action is needed?

Everyone’s natural inclination is to demand more output. More ink. They have more and we have less so spit out more. Business is geared to numbers continuously going up. You can throw as much explanation and caveats around a dip in a chart, but all the client will see is that it’s going down and down is bad. The competition is going up and up is good.

“But what if the rival’s boost occurred because their CEO drop-kicked a puppy?”

But what if the rival’s boost occurred because their CEO drop-kicked a puppy? What if their product was suddenly uncovered to be dangerous? What if their factories just burned down and there is endless discussion as to whether they will be able to survive the quarter? Would we recommend our client to seek more coverage just to match this?

Of course we wouldn’t. So that brings me back to the question, what information does share of voice provide that guides an action? What action can you take based on a SOV metric alone? And if SOV alone can’t guide a decision the way sentiment, or quality of coverage, or even volume of coverage can … then is it a metric we want to be using prominently?

The more I examine it, SOV as a metric distracts from the outcomes, is potentially misleading in and of itself, and provides little information value relative to the resources required to collect it.

What our clients are not properly asking for when they say “show me our share of voice” is “mindshare” or what they truly care about which is “share of wallet.”  They want to know what the perception of their brand is in relation to other brands. This is not data that you can collect through counting volume of clips or mentions. This is not volume of coverage but a measure of top of mind awareness. A measure of how much of a family’s resources get devoted to our client’s offerings. A research effort in and of itself.

It would seem to me that SOV as we’re currently looking at it is useful only in situations where we know a PR spend was on par with the competition (say in a sponsorship situation) or as part of an initial audit of the landscape to see how people are discussing brands relative to one another and where media bias towards one brand or another may exist.

But I would appreciate input and thoughts; the wisdom of the crowd. What say you all? Am I tampering with forces man was never meant to tamper with? Will they call me mad at the academy?

Comments or angry tweets below, or to @theelusivefish.

[About the author: Rob Clark is the Director of Insights and Measurement in the Digital practice in Edelman’s Toronto office, a wearer of funky ties and all-round smart guy. You can follow him on Twitter at @theelusivefish.]

Don’t Be Fooled By Last-Click Analysis Of Social Media

Forrester recently published a report entitled “The Purchase Path of Online Buyers.” Normally I’m a fan of Forrester’s reports, but this one left me scratching my head.

The report looks at transaction data from 15 clients of a marketing agency (let’s ignore that built-in bias, and convenient product placement in the recommendations, for the sake of this post) to draw conclusions about buyer behaviour including:

  • Most buyers do not arrive at a site directly — they come from search or other marketing activities (fair)
  • Last-click measurement is insufficient – it works for email and search but other tactics receive insufficient credit “as they are typically early in the research funnel and are followed by visits to search engines or email” (fair)
  • Email was effective during key promotional dates — not surprising, as retailers engage in heavy email drives during those times (think of Thanksgiving or Boxing Day)
However, one conclusion stood out to me:
“Hope for the best, but expect the worst with social.”

(Not surprising that it caught my eye, huh?)

The more I thought about, it the more I was left confused at Forrester’s characterization of the data, especially given the earlier warning about last-click measurement.

Where to begin?

No detail in the methodology

The report doesn’t actually give any detail as to the form the data that was used took. How did they track referrers, especially those two levels deep? Could they identify traffic that came from mobile social apps or from popular desktop apps such as TweetDeck, etc? From the methodology it sounds like they used the agency’s own models to estimate the data, but really we have no idea.

Zero detail on actual social media activities

At no point in the report does it say that (a) the companies were engaged in any sort of social media activities, (b) the form that those activities took (c) the scale and reach that those activities had, or (d) the quality of those activities.

How are we supposed to just agree with the conclusion that social media drove minimal sales if we don’t know what form that social media took, if any? It’s like saying “media relations drove minimal sales” without saying whether the organization actually did any media relations.

Hell, the numbers could actually be a good thing - if I wasn’t engaged in any form of social media but it still drove 2% of my sales, that might actually be a sign that I should begin to invest in it.

Social media objectives vary

In my opinion, brands make a mistake when they consider social media as purely conversion-driven tools. Social media provides numerous potential benefits besides end-of-funnel conversion, including:

  • Long-term brand-building
  • Top-of-mind awareness
  • Improved customer service and retention
  • Market intelligence and insights

Social media isn’t just bottom-of-the-funnel

Related to the last point, companies should consider the average person’s mindset when they’re using social media.

When you’re using Facebook or Twitter, for example, are you often looking for a link to take me to the best place to purchase something? Probably not. If you’re in a purchase-focused mindset, you’re more likely to be looking for reviews or recommendations from other people.

Once you receive them, maybe you’ll dig around for product information on the recommendation, then look for somewhere to buy it.

For example, today I asked Twitter whether I should buy a LiveScribe pen so I could capture my notes in Evernote:



Sure enough, I got a bunch of points of view in return:

Note: None of those responses offer a way to click through to purchase. Still, even though these results affected my likelihood to purchase, if I were to purchase the pen (which I may), a search engine would likely get the credit — even though search has done nothing to influence my actions up to this point — as I would use Google to find the company’s website.

Forrester seems to be ignoring its own warning that tactics other than email and search may be under-attributed, and passing judgement without fully considering the context.

Social media isn’t one-dimensional

This Forrester report focused on online buying behaviour, so it might seem harsh to criticize it for its single-minded focus. That’s the problem, though — when you’re dealing with a complex, multi-functional set of tools like social media, considering a single dimension in isolation from the others risks writing-off broad benefits across the organization.

This only reinforces the need for a broader focus than an eight-page report can produce – one looking at the effects of integrated, multi-disciplinary approaches to social media. Analyzing an inherently integrated, multi-dimensional set of tools in any other way leads to an incomplete picture.

(Image: talltomz)

Digital Communications Conversations Continue To Evolve

The maturation of the conversation about social media-driven PR continues.

As the “digital communications” industry (read: somewhere in the intersection of owned, earned and paid media) grows beyond the niche and towards the mainstream, the conversation at the leading end of the industry is shifting more and more from the “why” to the “how”.

Brian Solis published a great post yesterday, touching on a couple of important topics at the heart of the evolution of social media-focused communications from shiny objects to a serious business tool:

  • The continued focus of the metrics conversation on conversion as an endpoint
  • The definition of “influencers” from a client perspective

Conversion

More and more people are honing in on the importance of driving towards tangible, credible metrics. Conversions are key within that bracket. What you define as a conversion may vary depending on context, but in general the term will focus you in on sales, memberships, opt-in contacts (leads) and the like.

Why do conversions matter? Because they’re metrics that can lead to financial results (or could actually measure that themselves in the case of sales).

Folks like KD Paine, Olivier Blanchard and Rob Clark, who are neck-deep in the topic, figured this stuff out a while back but we’re slowly seeing more people hone in on it. Meanwhile, they’re focusing on how to prove value over “what you should do” and “why you should do it.”

As Chris Brogan put in a well-timed post on social media metrics today, “The social media metric that I think does matter and that is difficult to fully qualify is sentiment: the positive or negative mentions of a brand, product, service, whatever.” While unlike him I choose not to “poo-poo” measuring things like retweets, page views etc, I completely agree that in general they’re at best a proxy towards measuring more useful results, and at worst a cop-out from people unsure how to measure final results or afraid what it would show.

Influencers

Research continues to highlight the importance of context over quantity when it comes to influence. As Solis puts it:

“Brands seeking reach, presence, and connectivity must look beyond popularity and focus on aligning with the influential beacons who serve as the hubs for contextual networks or nicheworks.”

This is becoming more and more important as the early adopters who have built large followings become the focus of more and more attention from marketers – not only will the returns be greater from focusing on niche influencers in terms of success ratio; those people have more credibility within those communities.

Simply put, while I’m likely to check out a new business contact management tool if Brogan recommends it, if he suddenly recommended a lawnmower (for example), I’m much less likely to listen than if he were a credible and regular voice on that topic. That’s not a slight on him, but a reflection of the credibility that people can build up in a community that cares about their niche – that’s why sites like Pulse of the Tweeters help to drive things forward a notch.

So, the conversations on social graphs and content topics continue to converge.

Follow these trends; they’re important ones for anyone working in the digital communications space.

Sysomos Audience Moves Towards Measuring Social Media ROI

Social media ROI is a hot topic right now, as social media begins to (slowly) mature. The purists who insisted that the conversation alone was and end, rather than a means, are diminishing in volume and a more rational, approach is emerging balancing the revolutionary aspects of social media with those that are simply evolutionary from existing business practices.

One area in particular which is fast-evolving is social media monitoring (my ex-colleague Michael O’Connor Clarke quipped last week that there’s probably a micro-industry dedicated to watching it).

After several weeks of back and forth, and rescheduled meetings, I finally managed to get a demo of Sysomos Audience last week. I came away impressed.

Placing a Value on your Visitors

Sysomos Audience is an addition to the Heartbeat monitoring and engagement tool. At first glance it seems similar to Google Analytics in nature – in fact, I previously under the incorrect impression it simply connected social media traffic to web analytics. However, Audience really focuses in a different direction, providing tools that should pique the interest of your sales, marketing and community management folks alike.

Audience tracks visitors to your site alongside their previous web activity, and helps to determine whether each person is a real lead or is just browsing. It does so by examining peoples’ previous web activity, including competitors’ websites, blogs, social networks and so on. In doing so, it determines whether your visitors are qualified leads or just browsing. For example, people are much more likely to be serious sales leads if they’ve been researching other competitive products first than if they’ve just clicked through from a random site.

Critically, Audience also lets you assign a dollar value to visitors based on their visits to competitor sites, to help determine the ROI of your social media activities. It does so by letting you assign values for visits to different areas of your site (those key to your sales funnel might have a higher value, for example) and other factors. In doing so, you gain a relative value for each visitor to your site. This might seem familiar to web analytics (Google Analytics lets you assign goal values, for example) but this goes above and beyond by incorporating activities outside your own site, and by aggregating values per user.

This has implications for several functions within companies:

  1. Sales
  2. Community management
  3. Public relations

Sales

Sales folks – wouldn’t you like to know who your most valuable leads are right at the beginning of the process, so you can prioritize them accordingly? While Audience generally only provides generic tracking information for most people, if you hook the system into any web forms you have, it can link their name and information into their activities (note: you’ll likely need to amend your privacy policy in order to do this). Right now, the system doesn’t hook into Salesforce but according to Sysomos co-founder Nilesh Bansal, that functionality is on the way.

Community Management

Just as Audience lets you track your most valuable visitors, it also lets you identify the sites that are the source of the most valuable traffic to your website. In the demonstration I saw, for example, I saw that while TechCrunch drove a lot of traffic to Sysomos, the traffic from other sites on a per-user basis was actually worth more to them. For community managers, pulled in a thousand directions, this can be valuable information to help them prioritize their focus.

Public Relations

The idea of being able to place a value on the traffic from a piece of coverage is mouth-watering to me. For one, it gives a great answer to the “what’s the ROI of this pitch” question (which even traditional media relations hasn’t solved yet) but also it helps you to figure out who you need to build relationships with and on whom you should focus your pitching. Of course, it doesn’t remove the hands-on targeting and tailoring work that goes into each project, but this kind of data would still be immensely valuable.

Privacy Concerns?

The only question that worried me during the demo I received revolved around online privacy. How does Audience determine which sites people have visited recently? Every site I’ve seen reviewing Audience – from TechCrunch to ReadWriteWeb to Web Metrics Guru – have wondered but no answers are forthcoming. While Sysomos doesn’t currently pull user profiles in, it’s only a small step from there to linking a Twitter or Blogger profile into things and having a complete record of your visitors’ browsing habits. That’s hypothetical but a little concerning as I’m sure they’ll experience pressure to add that feature.

Sysomos’ Nilesh Bansal wouldn’t shed any light on the question when I spoke with him. He told me they don’t look at cookies, but that Audience uses a piece of JavaScript code which you embed on your site and correlates that with their social media monitoring database. So, how do they know people have been on a competitor’s site? It sounds a little dubious to me. As long as they don’t shed any insight into this, people will continue to wonder what’s going on.

Exciting Potential

Setting aside the privacy concerns for a moment, Audience really does have a lot of potential, especially if you’re already a Sysomos client. The product is still in closed beta testing for now and Sysomos hasn’t announced pricing but, like Radian6′s engagement console, this looks to be a differentiating addition to Sysomos’ portfolio of services. I do think they need to answer the privacy questions, though.

What’s your take?