Archive for May, 2011

The Little Give

11.05.30

Today I’m veering a little way from my normal topics – while I generally write about communications, social media and the like, I thought you might like a little insight into life at Edelman; specifically, one of the cool employee engagement initiatives we have here in Canada.

The Little Give

The Little Give is a CSR program, initially created by Edelman’s Vancouver office, that sees Edelman Canada offices split into teams and work to help charities in our communities.

This year, the Toronto office is supporting ten great organizations that work to improve the lives of youths in various ways. Everyone in the office was invited to participate in the nomination and voting process to decide the list of organizations, with the final list including:

(You can find out more about our partners over on our Little Give blog)

How it works

At 4pm on June 2nd, the Toronto office will split into ten teams, which will then work for the next 48 hours (we close the Edelman office for one day, and people donate a day of their weekends) to solve whatever problem their designated partner organization puts to them. Edelman seeds each team with $2,500 to use over the two days.

At the end of that time, each team comes back and gives a short presentation of what they did to the rest of the office and a panel of judges, who ultimately determine the winning project.

Why it works

Beyond a cheque: We could just hand over a cheque to these organizations, but by going beyond that and contributing 4,800 hours or so of expertise alongside the money, we can make a real difference to these organizations.

It’s authentic: As our General Manager Lisa Kimmel said, “CSR programs need to be authentic and clear and they need to be woven into the fabric of your corporate culture.” We’re not just handing over a cheque and moving on. We’re working to have a positive impact on our community.

Everyone’s engaged: From the selection of non-profits, to the fun run-up to the event (from the Little Give blog to the Facebook page to real-world in-office games), everyone in the office is involved, making this a high point of our employee engagement initiatives as well as a great CSR program. In fact, just last week a new employee mentioned how excited they were to join us in time for the event.

I narrowly missed-out on The Little Give last year, as I joined Edelman just after it happened. I’m totally psyched for this year’s.

 

Posted in: Agency life, miscellaneous

Tags:

2Comments

Compete Responds To Criticism

11.05.25

Last week I voiced some concerns around the traffic numbers that Compete.com reports for websites. My primary concern was that, after seeing nearly a 90% variance between their numbers and those from Google Analytics on my own site, I couldn’t trust that their numbers for other sites would be correct. This is important, as we often use Compete.com and similar sites to report on potential reach of coverage for our clients.

I was pleased to see that Compete responded, multiple times — in a comment on my post; via a tweet and then subsequently in email.

Given the interest I saw from other people, I obtained Compete’s permission to publish our exchange.

Comment on my original post:

Hi Dave – Matt here, Client Relations Director at Compete.com.  I am happy to do my best to clear up any confusion if you reach out to me at support@compete.com.  Compete is based on a sample of 2 million, US based panelists.  We have a small sample warning on your website, which means we have limited data on your domain.  Even when considering your GA numbers, you are well down the long tail of the internet.  The code you’ve installed, it is just for audience profiles and it is only found within that tab of the product.  It does not impact any traffic numbers.  In any event, I’d be happy to explain more if you would like to reach out.

While I (and several people on Twitter) wasn’t thrilled with the tone of “you are well down the long tail of the internet” (which sends a message of “just wanted to remind you you’re nothing to us”), I took Matt up on his offer and emailed their support address:

Hi,

I’m reaching out following a tweet from your twitter account regarding a recent blog post of mine (http://t.co/lCXr4UN).

I’d love to understand more about why the numbers seem to diverge so much from what I’m seeing on my website logs.

Thanks,

Dave

Sure enough, a day later I received a reply:

Hi Dave,

Thanks for reaching out to us about your site. I know that it can often be confusing [Dave: no, it's not confusing; it's just irritating. Moving on...] when comparing the traffic numbers for local analytics (google, omniture, etc) to the numbers on Compete.com. At the core the methodologies are like comparing apples to oranges, they’re both fruit – just produced from different trees. Think of compete numbers as an orange – a U.S. based research numbers that help you understand your size and trends against your competition. Local analytics is more like the apple that helps you understand what’s happening on your site so you can improve your visitor’s experience. They’re great supplements to one another in terms of getting a more complete picture of the internet, but are inherently very different in the approach you take to consuming the data sets.

From a more technical perspective panel-based clickstream data (Compete.com) and web analytics data (local analytics (Google Analytics) and server logs) stems from the underlying methodologies that each approach use. At a high-level, panel-based providers like Compete measure online behavior based on consumers, whereas local analytics measure similar behaviors based on cookies. The consumer metrics that panel companies provide are based on statistically-derived estimates that are derived from a representative sample of consumers; in this instance, the behaviors of the sample are weighted and extrapolated to represent the entire internet browser population. The cookie-centric metrics that web analytics companies provide are developed on simple counts of cookies for all of the web pages that are a tagged on a site or a set of sites; when a consumer visits a specific page on a site, that visit is counted by the web analytics platform.

Both approaches have their strengths and limitations. Panel-based measurement provides excellent insight into visitor demographics, what consumers do across all of the websites they visit and analysis over long time periods. The limitation of consumer panels is that they sometimes do not provide sufficient sample to measure “low incidence” behaviors such as visiting very small sites, using rare search terms, or interacting with low-traffic pages on specific websites. Compete’s panel is one of the largest in the industry, this helps us ensure that we can measure and report on more of these infrequent behaviors compared to other panel providers.

If you take a look at information which is collected in a similar fashion below you’ll see either no data or considerably smaller numbers than local analytics.
http://trends.google.com/websites?q=davefleet.com&geo=all&date=all
http://www.quantcast.com/davefleet.com

Cookie-based web solutions are good sources of information on all of the behaviors that occur within a website, and therefore can be used to calculate and optimize site flow, conversion rate and other onsite activities. In this manner, web analytics are not subject to the same sample requirements as panel companies. However, there are some limitations that cookie-based solutions are susceptible to that panel-based measurement services are not. Cookie-based data can be affected and sometimes inflated by the deletion of a user’s cookies, incorrect page tagging, and susceptibility to bots or spiders. Also, the data found on Compete.com comes from our panel of U.S. users, local analytics and server logs collect U.S. and International data.

I hope this explanation can help clear up any questions that you might have with our numbers vs. those seen when looking at Google Analytics and server logs. Please let me know if you have any other questions that I can help you answer. If you’d like to learn more about our methodology please reference our data methodology whitepaper: http://media.compete.com/site_media/upl/img/Compete%20Data%20Methodology.pdf.

Kind Regards,

Lindsey
Compete.com Customer Support

While some people I spoke to felt the answer was a bit condescending, I thought it did a good job of explaining the difference in methodologies between Compete and Google Analytics in plain language. Ok, so now I understood the difference between the two services’ methodologies (I didn’t ask, but it’s helpful nonetheless). The question remained, though – why should people trust Compete to provide data on anything but the top tier of sites on the Internet?

So I asked:

Hi, [I didn't realize until afterward that the original email had been signed -- my bad]

Thanks for the thoughtful response.

I do have two follow-up questions for you:

1. For sites that are, in the words of your Client Relations Director, “well down the long tail of the internet” [Dave: I couldn't resist], do you therefore recommend using other data sources than Compete?
2. Would you mind if I published your response as an update to my blog post? I would love to include your side of the story for people to consider.

Thanks,

Dave

Compete’s response:

Hi Dave,

Thanks again for reaching out, we’re happy to have you publish our dialog and are open to answering your questions about our tools and how it relates to the industry. Though I would request that you include your follow-up question to provide context to our reply.

In terms of data sources, our client relations team advocates for two things. First, a firm understanding of the data you’re looking at through asking targeted questions about the data source. When you’re unfamiliar with a data set this can be really difficult, because you don’t know what you don’t know, right? The second challenge is using the proper tool for the job, compete specifically isn’t meant to replace your local analytics – this is not our goal.

Compete ultimately is looking to provide our customers with a better understanding of their industry and competitors from a research perspective. If your data set on our site is listed under a small sample warning, the data will have more of a directional relevance that gives you an idea of what’s going on for the industry. For sites with more traffic activity, our data is really helpful in understanding the approach competitors take in terms of SEO, SEM, and traffic acquisition to name a few popular insights typically gleaned.

If you take a look at the image attached (Blogging services 2 year category view*) you’ll notice that between the month of March and April there is a slight monthly decrease of about -2.7%. Looking at the trend in 2010 the dip was about -1.1%, so this 2 year view allows you to see that there’s something happening within US online consumer’s behavoir that makes it so that each year between March and April there is a slight dip in the amount of activity to the overall category (which includes domains like blogger and wordpress).

*Compete Categories are groups of domains we organize for our PRO and Enterprise Level Subscribers.

From my personal go-to toolkit, my data sources could include Google Trends – think search data, similar sites, and geo-demographics. If you’re looking for sentiment data visa-vi social networks, I like using Google Realtime to build out timelines that correspond with news and product happenings. I use these services a lot to supplement our information, they’re not a direct replacement in terms of the value add for a lot of digital marketers but they can help give you a better picture of what people were taking about during specific time periods, breaking news, and similar domains that may be of interest to investigate.

When it comes to studying the internet and ultimately the behavoir of users, getting familiar with new tools and services can be a difficult process if you don’t have the resources for full data immersion. If you’re looking to attempt to prompt specific actions you’d like visitors to take, it takes time and practice. One of the things that Avinash* typically preaches, and I like to echo, is that marketers should always be aiming to synthesize what the data trends indicate, rather than simply “reporting”.

Here’s the post* http://www.kaushik.net/avinash/2011/04/difference-web-reporting-web-analysis.html

I hope this helps, please let us know if you have any other questions.

Kind Regards,

Lindsey

A couple of thoughts from my end:

Firstly, thank you to Matt and Lindsay from Compete for their thoughtful responses, and for allowing me to publish this exchange. I appreciate the thought and the time spent on their end.

Secondly, it appears that Compete doesn’t intend for its traffic numbers to be used for analysis – from their response, it appears that “For sites with more traffic activity, our data is really helpful in understanding the approach competitors take in terms of SEO, SEM, and traffic acquisition” insights are the primary uses for their data. Fair enough – I hadn’t thought of Compete in that way before, and it’s good to know that that’s their intent.

Thirdly, I still have two outstanding questions.

Outstanding question #1: How accurate are Compete’s/Quantcast’s/Alexa’s numbers for top-tier websites?

Are we looking at a 5% error margin? 15%? 25%? I’d love to know, because we have a duty to clients to know how accurate the numbers we’re using are.

Compete’s team, for all heir helpfulness, still hasn’t explained why people should trust their traffic numbers for sites (although, frankly, I could have been clearer as to why I was asking). This is still a critical issue – to quote my original post:

Should I believe that CNN.com’s traffic went up by 27% in March compared to February? Should I believe that Mashable’s traffic went down by nearly 30% in the last year?

Why is it important? Because I and many other people look at Compete’s numbers to determine sites’ traffic numbers when reporting on the results of our activities. If we can’t believe those numbers, we need to look elsewhere.

Outstanding question #2: What is the best site – free or paid – for providing reach analysis of lower-tier websites?

I readily acknowledge that Compete is a free service (it doesn’t sound like their Pro service adds much in terms of accuracy – just longer time periods and additional data for analysis), and that perhaps I shouldn’t expect too much from a free service.

I’ll be clear, though: I would be happy to consider paid services if they’re able to offer accurate reports.

Let’s face it – few companies are able to conduct outreach targeting only top-tier websites. Especially when you get into niches, there are relatively few relevant sites with traffic comparable to the top-tier of the Internet. So, where do we go for analysis of the rest?

What do you think? My questions again:

  1. How accurate are Compete’s/Quantcast’s/Alexa’s numbers for top-tier websites?
  2. What, in your opinion, is the best site – free or paid – for providing reach analysis of lower-tier websites?

Posted in: measurement, tools

Tags: ,

1Comments

Is Share of Voice a Useless PR Metric?

11.05.24

This is a guest post by my Edelman colleague Rob Clark

Sometimes you say a word too many times in a row and the word slowly begins to lose meaning for yourself. It becomes foreign gibberish and you begin to wonder if you’re pronouncing this thing correctly or if it was ever really a word at all. Which is all to say that I’ve been giving a lot of thought lately to share of voice (SOV) and I may have passed the threshold where it ceases to hold meaning.

Why do we measure?

We measure because there is a decision we have to make and we are lacking the data needed to take action. So I would like to ask what information does share of voice provide the PR practitioner that guides an action?

In marketing – where all is a funnel down from eyeballs to wallets and the space and time is finite – SOV provides insight into whether your message is drowned by the competition’s. But the real strategic advantage comes in that the cost of the ad space is a known quantity. Knowing what your competitor’s SOV in a market is, let’s you know what kind of resources they are pushing forth. You know which of their products is getting the thrust and in what markets. It shows you some of the cards they have on the table.

But in PR we don’t buy coverage by the pound. We can’t translate ink on the page (or pixels on the screen) into dollars spent on PR. So that set of data is lost to us from a SOV measure.

Editorial – though not infinite – is open to expand and contract. Your amount of coverage can remain consistent but your share contract tremendously as a flurry of write ups about your competitor come out. Let’s say that our client is Widget co. (makers of fine hypothetical examples since 1912). Widget co has a 20% SOV and their nearest competitor has 30%. The following month Widget co is at 18% and their rival at 37%. What decision will this info drive? What action is needed?

Everyone’s natural inclination is to demand more output. More ink. They have more and we have less so spit out more. Business is geared to numbers continuously going up. You can throw as much explanation and caveats around a dip in a chart, but all the client will see is that it’s going down and down is bad. The competition is going up and up is good.

“But what if the rival’s boost occurred because their CEO drop-kicked a puppy?”

But what if the rival’s boost occurred because their CEO drop-kicked a puppy? What if their product was suddenly uncovered to be dangerous? What if their factories just burned down and there is endless discussion as to whether they will be able to survive the quarter? Would we recommend our client to seek more coverage just to match this?

Of course we wouldn’t. So that brings me back to the question, what information does share of voice provide that guides an action? What action can you take based on a SOV metric alone? And if SOV alone can’t guide a decision the way sentiment, or quality of coverage, or even volume of coverage can … then is it a metric we want to be using prominently?

The more I examine it, SOV as a metric distracts from the outcomes, is potentially misleading in and of itself, and provides little information value relative to the resources required to collect it.

What our clients are not properly asking for when they say “show me our share of voice” is “mindshare” or what they truly care about which is “share of wallet.”  They want to know what the perception of their brand is in relation to other brands. This is not data that you can collect through counting volume of clips or mentions. This is not volume of coverage but a measure of top of mind awareness. A measure of how much of a family’s resources get devoted to our client’s offerings. A research effort in and of itself.

It would seem to me that SOV as we’re currently looking at it is useful only in situations where we know a PR spend was on par with the competition (say in a sponsorship situation) or as part of an initial audit of the landscape to see how people are discussing brands relative to one another and where media bias towards one brand or another may exist.

But I would appreciate input and thoughts; the wisdom of the crowd. What say you all? Am I tampering with forces man was never meant to tamper with? Will they call me mad at the academy?

Comments or angry tweets below, or to @theelusivefish.

[About the author: Rob Clark is the Director of Insights and Measurement in the Digital practice in Edelman's Toronto office, a wearer of funky ties and all-round smart guy. You can follow him on Twitter at @theelusivefish.]

Posted in: communications, measurement, pr, public relations

Tags: , , ,

19Comments

Compete.com Doesn’t Compete

11.05.18

I can’t take it any more. For years I’ve suffered in (relative) silence, while we’ve continued to use a tool that doesn’t seem to reflect any form of reality, yet no-one seems to say anything about it. I need to ask:

Why do people still use Compete.com?

Case in point

Let’s take a look at my site – davefleet.com.

I’m not going to lie — I’ve neglected this poor thing recently, so traffic numbers have dropped… but Compete makes it look as though someone dropped a bomb on my server.

Let’s compare a few stats between Compete and Google Analytics, and see how things look.

Unique Visitors

  • Google Analytics: 14,972
  • Compete: 1,579 - an 89% variance

Visits

  • Google Analytics: 17,186
  • Compete: 1,900 – again, an 89% variance

“Ah but Dave, perhaps those Google numbers included bots,” I hear you say. Well, I heard myself say it too, so I thought I’d take a closer look.

That’s 97% of visits from browsers – from IE, Firefox, Chrome or Safari. My analytics also tell me that nearly 60% of traffic to the site comes from search engines.

So, where does that leave Compete. Maybe if I gave them access to my site, they’d be able to give more accurate numbers, right?

Wrong.

I have the Compete.com analytics code installed on my site, yet it still spills-out this garbage.

If I can’t trust Compete to get the numbers right when its own code is installed, how can I trust it to get it right when I’m looking at other sites? The numbers it reports border on bizarre, yet we just nod and keep on using it.

Should I believe that CNN.com’s traffic went up by 27% in March compared to February? Should I believe that Mashable’s traffic went down by nearly 30% in the last year?

*twitch*

I think I need to lie down now.

What tools do you use to estimate traffic on third party sites?

Posted in: tools

19Comments

Don’t Be Fooled By Last-Click Analysis Of Social Media

11.05.16

Forrester recently published a report entitled “The Purchase Path of Online Buyers.” Normally I’m a fan of Forrester’s reports, but this one left me scratching my head.

The report looks at transaction data from 15 clients of a marketing agency (let’s ignore that built-in bias, and convenient product placement in the recommendations, for the sake of this post) to draw conclusions about buyer behaviour including:

  • Most buyers do not arrive at a site directly — they come from search or other marketing activities (fair)
  • Last-click measurement is insufficient – it works for email and search but other tactics receive insufficient credit “as they are typically early in the research funnel and are followed by visits to search engines or email” (fair)
  • Email was effective during key promotional dates — not surprising, as retailers engage in heavy email drives during those times (think of Thanksgiving or Boxing Day)
However, one conclusion stood out to me:
“Hope for the best, but expect the worst with social.”

(Not surprising that it caught my eye, huh?)

The more I thought about, it the more I was left confused at Forrester’s characterization of the data, especially given the earlier warning about last-click measurement.

Where to begin?

No detail in the methodology

The report doesn’t actually give any detail as to the form the data that was used took. How did they track referrers, especially those two levels deep? Could they identify traffic that came from mobile social apps or from popular desktop apps such as TweetDeck, etc? From the methodology it sounds like they used the agency’s own models to estimate the data, but really we have no idea.

Zero detail on actual social media activities

At no point in the report does it say that (a) the companies were engaged in any sort of social media activities, (b) the form that those activities took (c) the scale and reach that those activities had, or (d) the quality of those activities.

How are we supposed to just agree with the conclusion that social media drove minimal sales if we don’t know what form that social media took, if any? It’s like saying “media relations drove minimal sales” without saying whether the organization actually did any media relations.

Hell, the numbers could actually be a good thing - if I wasn’t engaged in any form of social media but it still drove 2% of my sales, that might actually be a sign that I should begin to invest in it.

Social media objectives vary

In my opinion, brands make a mistake when they consider social media as purely conversion-driven tools. Social media provides numerous potential benefits besides end-of-funnel conversion, including:

  • Long-term brand-building
  • Top-of-mind awareness
  • Improved customer service and retention
  • Market intelligence and insights

Social media isn’t just bottom-of-the-funnel

Related to the last point, companies should consider the average person’s mindset when they’re using social media.

When you’re using Facebook or Twitter, for example, are you often looking for a link to take me to the best place to purchase something? Probably not. If you’re in a purchase-focused mindset, you’re more likely to be looking for reviews or recommendations from other people.

Once you receive them, maybe you’ll dig around for product information on the recommendation, then look for somewhere to buy it.

For example, today I asked Twitter whether I should buy a LiveScribe pen so I could capture my notes in Evernote:



Sure enough, I got a bunch of points of view in return:

Note: None of those responses offer a way to click through to purchase. Still, even though these results affected my likelihood to purchase, if I were to purchase the pen (which I may), a search engine would likely get the credit — even though search has done nothing to influence my actions up to this point — as I would use Google to find the company’s website.

Forrester seems to be ignoring its own warning that tactics other than email and search may be under-attributed, and passing judgement without fully considering the context.

Social media isn’t one-dimensional

This Forrester report focused on online buying behaviour, so it might seem harsh to criticize it for its single-minded focus. That’s the problem, though — when you’re dealing with a complex, multi-functional set of tools like social media, considering a single dimension in isolation from the others risks writing-off broad benefits across the organization.

This only reinforces the need for a broader focus than an eight-page report can produce – one looking at the effects of integrated, multi-disciplinary approaches to social media. Analyzing an inherently integrated, multi-dimensional set of tools in any other way leads to an incomplete picture.

(Image: talltomz)

Posted in: measurement, social media

Tags: ,

9Comments

The Non-Review Review: The Now Revolution

11.05.02

A quick confession for you: about six months ago I snagged a pre-release copy of the book “The Now Revolution,” by my friends Amber Naslund and Jay Baer. I quickly read it, absorbed it and loved it.

The Now RevolutionThen I got busy… really busy… and failed to find time to write a review of the book. Over time, my memory of the specific highlights faded and it became more and more futile to try to write the review without re-reading the book.

I still intend to re-read the book, and will eventually write the review, but for now know this:

  • I loved The Now Revolution
  • It covers social strategy at a high level, from philosophy to business culture to listening and more
  • I constantly recommend it to people, and I don’t do that lightly
  • I recommend you pick up a copy if you haven’t already

I’ll write more when life is calmer.

Posted in: books

Tags:

5Comments