Don’t Be Fooled By Last-Click Analysis Of Social Media

Forrester recently published a report entitled “The Purchase Path of Online Buyers.” Normally I’m a fan of Forrester’s reports, but this one left me scratching my head.

The report looks at transaction data from 15 clients of a marketing agency (let’s ignore that built-in bias, and convenient product placement in the recommendations, for the sake of this post) to draw conclusions about buyer behaviour including:

  • Most buyers do not arrive at a site directly — they come from search or other marketing activities (fair)
  • Last-click measurement is insufficient – it works for email and search but other tactics receive insufficient credit “as they are typically early in the research funnel and are followed by visits to search engines or email” (fair)
  • Email was effective during key promotional dates — not surprising, as retailers engage in heavy email drives during those times (think of Thanksgiving or Boxing Day)
However, one conclusion stood out to me:
“Hope for the best, but expect the worst with social.”

(Not surprising that it caught my eye, huh?)

The more I thought about, it the more I was left confused at Forrester’s characterization of the data, especially given the earlier warning about last-click measurement.

Where to begin?

No detail in the methodology

The report doesn’t actually give any detail as to the form the data that was used took. How did they track referrers, especially those two levels deep? Could they identify traffic that came from mobile social apps or from popular desktop apps such as TweetDeck, etc? From the methodology it sounds like they used the agency’s own models to estimate the data, but really we have no idea.

Zero detail on actual social media activities

At no point in the report does it say that (a) the companies were engaged in any sort of social media activities, (b) the form that those activities took (c) the scale and reach that those activities had, or (d) the quality of those activities.

How are we supposed to just agree with the conclusion that social media drove minimal sales if we don’t know what form that social media took, if any? It’s like saying “media relations drove minimal sales” without saying whether the organization actually did any media relations.

Hell, the numbers could actually be a good thing – if I wasn’t engaged in any form of social media but it still drove 2% of my sales, that might actually be a sign that I should begin to invest in it.

Social media objectives vary

In my opinion, brands make a mistake when they consider social media as purely conversion-driven tools. Social media provides numerous potential benefits besides end-of-funnel conversion, including:

  • Long-term brand-building
  • Top-of-mind awareness
  • Improved customer service and retention
  • Market intelligence and insights

Social media isn’t just bottom-of-the-funnel

Related to the last point, companies should consider the average person’s mindset when they’re using social media.

When you’re using Facebook or Twitter, for example, are you often looking for a link to take me to the best place to purchase something? Probably not. If you’re in a purchase-focused mindset, you’re more likely to be looking for reviews or recommendations from other people.

Once you receive them, maybe you’ll dig around for product information on the recommendation, then look for somewhere to buy it.

For example, today I asked Twitter whether I should buy a LiveScribe pen so I could capture my notes in Evernote:

Sure enough, I got a bunch of points of view in return:

Note: None of those responses offer a way to click through to purchase. Still, even though these results affected my likelihood to purchase, if I were to purchase the pen (which I may), a search engine would likely get the credit — even though search has done nothing to influence my actions up to this point — as I would use Google to find the company’s website.

Forrester seems to be ignoring its own warning that tactics other than email and search may be under-attributed, and passing judgement without fully considering the context.

Social media isn’t one-dimensional

This Forrester report focused on online buying behaviour, so it might seem harsh to criticize it for its single-minded focus. That’s the problem, though — when you’re dealing with a complex, multi-functional set of tools like social media, considering a single dimension in isolation from the others risks writing-off broad benefits across the organization.

This only reinforces the need for a broader focus than an eight-page report can produce – one looking at the effects of integrated, multi-disciplinary approaches to social media. Analyzing an inherently integrated, multi-dimensional set of tools in any other way leads to an incomplete picture.

(Image: talltomz)