Advancing communications measurement and evaluation

How to Choose Tools and Benchmark for Social Media Measurement Using the AMEC Valid Metrics Framework

Boy listens to social media.

This article is Part 9 of an ongoing series, “Rescuing Ourselves from Social Media Measurement Dinosaur-dom,” based on Angela Jeffrey’s paper “Social Media Measurement: A Step by Step Approach.” It’s a free download and includes much more detail than this series does. For another helpful resource, view this video guide to using the AMEC Social Media Framework, complete with case study.

Today we wade deeper into Step Five of my Eight Step Social Media Measurement Process: “Choose Tools and Benchmark using the AMEC Valid Metrics Framework.”

  The Eight-Step Social Media Measurement Process

  1. Identify organizational and departmental goals.
  2. Research stakeholders for each and prioritize.
  3. Set specific objectives for each prioritized stakeholder group.
  4. Set social media Key Performance Indicators (KPIs) against each stakeholder objective.
  5. Choose tools and benchmark (using the AMEC Matrix).
  6. Analyze the results and compare to costs.
  7. Present to management.
  8. Measure continuously and improve performance.

Last time, I introduced you to the overall AMEC Valid Metrics Framework in its original form, along with a newer Social Media Framework. This time we’ll go into the tools to get the job done!

Please note: Each stage in the process described in this article is backed up by an enormous and very useful appendix of tools and metrics in the full paper here: “Social Media Measurement: a Step by Step Approach.”

And now Step Five Continues… Choose Tools and Benchmark (using the AMEC Valid Metrics Framework)

Let’s start with a quick review of the Valid Metrics Framework.

The Valid Metrics Framework is a visual model of measurement applied to several areas of PR that helps practitioners understand the measurement process and choose effective metrics for each step in that process (see the example below, for Social/Community Engagement). It breaks down the practice of public relations into three phases, which run along the vertical axis:

  • Public Relations Activity – Metrics that measure efforts in producing and disseminating messages.
  • Intermediary Effect – Metrics that measure third-party dissemination of messages to target audiences.
  • Target Audience Effect – Metrics showing the target audience has received the messages and any resulting action-driven outcomes.

The horizontal axis reflects five communications or marketing stages, which progress from initial audience awareness to audience action:

  • Awareness – Are intermediary and target audiences aware of campaign efforts?
  • Knowledge – Are these audiences becoming clearer about the facts?
  • Interest – Are they giving consideration to an organization’s offer?
  • Support – Have they moved toward supporting the offer?
  • Action – Are they taking action in measurable business outcomes?

Valid Metrics Framework for Social/Community Engagement

The Valid Metrics Framework for Social/Community Engagement

In the example above, suitable metrics have been grouped to help demonstrate a campaign’s success vertically (from simple activity to output and outcome results) and horizontally (through the five communications stages).

Your best-practice measurement goal is to move diagonally across the matrix, from top left to bottom right, to the furthest degree possible. Keep in mind: Only choose a couple metrics for the areas of the Framework you are concerned with, and just keep focusing on the Target Audience Effect.

Choosing Metrics to Measure Each Stage

Public Relations Activity

What actual PR activities will be undertaken to build a program? It’s critical to keep track of these efforts by monitoring the types of activities the example above suggests, such as:

  • Content creation (the number of assets created, videos/podcasts);
  • Social media engagement (numbers of blog posts, blogger events, blogger briefings, Twitter posts, community site posts and events);
  • Influencer and stakeholder engagement (what activities were undertaken to drive engagement forward, such as the number of Facebook and Twitter posts, community site posts, etc.); and
  • Events/speeches, offline community events, and traditional media outreach.

These data points can later be compared to resulting response and business activity, and can even be correlated to business results. However, at this point, they do not measure actual results of the campaign, they simply measure campaign efforts.

Intermediary Effects

This is where a practitioner ascertains how the media, bloggers, and key influencers (third parties) responded to PR activities across the horizontal axis of Awareness, Knowledge, Interest, and Support. Are they communicating the right messages, in a positive manner, in a greater volume than competitive messages? Have they gone so far as to endorse or recommend? The goal for Intermediary Effects is to get past the simplistic quantitative measures in the Awareness column (which are really just proxies for potential awareness), and toward the more sophisticated metrics in Interest and Support.

There are two basic methods for measuring Intermediary Effects:

  1. Measuring Owned Media Sites – websites, Facebook, and Twitter pages, for example.
  2. Measuring Non-Owned Sites, a.k.a. “earned media,” which includes everything else.

One caveat for this section: It is often very difficult, if not impossible, to determine if the responders to owned and earned media are really intermediary channels or are, in fact, target audience members in social media. That is exactly why AMEC has developed an updated Framework (as featured in my previous installment in this series). However, the measures shared below are effective regardless of whether an intermediary third party or a target audience member responded to campaign activities.

1. Measuring Owned Sites

The first step is to benchmark and set measures for each of an organization’s or department’s owned sites, be they websites, blogs, Twitter, Facebook pages, YouTube, etc. Pick and choose the metrics that can best act as KPIs for program objectives and benchmark before the program begins. The following is just an outline, but Appendix B (in the full paper) includes a full selection of metrics and tools that summarize key advice for blogs and websites:

Tools for Measuring Blogs and Websites
(This is an excerpt from Appendix B in the full paper. It’s included here as an example of the large number of resources listed in the paper itself.)

Visitors, Unique Visitors and Visits – Benchmarking these simple numbers for owned sites is important for gauging awareness as intermediaries and target audiences move across the funnel. They can be found in Google Analytics, which is free, or in other paid tools. Google has recently expanded its offerings through NextAnalytics for Excel, which offers hundreds of charts aggregating one’s website, Facebook page, Twitter page, LinkedIn, YouTube and other data feeds. The software packages range from $299/year to $995/year and deliver data into Excel spreadsheets, charts or graphs. NextAnalytics is definitely worth checking out.

Web analytics tracks movement from basic exposure/awareness through all the steps of the AMEC Valid Metrics Framework, and can complete the picture with conversions to sales, memberships, page downloads, and dozens of other metrics. To identify Intermediary Activity, look at referral sites to see which media sources, blogs or influencers are sending traffic.

Comments – Counting comments is also important for gauging awareness, but, to measure further across the Framework, consider using Stowe Boyd’s Conversation Index, which measures the # of Audience Comments or Replies per Post (divided by the total posts or activity). If an individual is posting more often than he is receiving comments, he is in a broadcasting stage and his audience is not progressing through the stages. For more on this, read Measuring the Online Conversation, by Kami Huyse (2006).

Site and Search Rankings and Authority – Where does an organization’s site show up in a Google search? How does it show up in AlexaCompete, or Quantcast rankings? What about inbound links? Or Google PageRank? Is all this information available from a single free site? Yes: WMTips.com provides it in the Site Information Tool. This site also provides a great list of the top keywords. Again, all are great metrics to benchmark before a program begins and then track along the way.

Linkbacks – How many people have clicked-through on blog posts or Twitter links? To find out, make sure to use a URL-shortening site like Bit.ly, which tracks the number of clicks on the link. Hootsuite also provides link tracking as one of its free report options. Jim Sterne (2010, Social media metrics: How to measure and optimize your marketing Investment. Hoboken, N.J.: John Wiley & Sons, Inc.) advises coding each link by a unique code that identifies the campaign. To do this, simply add a “?” mark to the end of each link followed by a campaign identifier. Then, when they are re-tweeted, they can be tracked. In case a campaign is already in progress, and posts did not use Bit.ly links, it is still possible to get some idea of how many people linked to those posts by going to Google and entering “link:URL.” This method usually undercounts links, but it is a start. Also see Open Site Explorer by SEOmoz.

The items below are included in Appendix B (in the full paper), as well.

  • Twitter Sites
  • Facebook Sites
  • Bookmarking Sites
  • YouTube, Flickr and Other Image Sites
  • Compound Influence Scores

Track the growth on all the channels mentioned above so you can eventually overlay the scores with PR activities, correlate the two, and identify what is working best.

2. Measuring Non-Owned Sites, a.k.a. Earned Media

Next, see what Intermediary Effects the media and bloggers are having on the campaign. First, find a good listening platform to bring in as much content as possible. A few free (or nearly free) tools include Google, SocialMention, Twazzup, HootSuite and IceRocket. Some great paid tools include Vocus, SocialEyez, Visible Technologies, Sysomos, SDL Social Intelligence Solutions, and Moreover NewsDesk.

Once the content is obtained, determine which KPIs work best to indicate movement between Awareness, Knowledge, Interest, and Support. These include: Share of conversation, key message alignment, accuracy of facts, expressed opinions, endorsements by journalists or influencers, and rankings on industry lists.

Two key methods for measuring earned media include the following, which are presented in detail in Appendix C (in the full paper):

  • Content Analysis,  for qualitative evaluation of what is being said; and
  • Source Strength,  for quantitative evaluation of earned-media sites.

Pulling metrics, regardless of how many are chosen, can take hours of work each week, especially if a number of free tools are used. It’s best to find a paid tool that fits campaign objectives.

Some great tips for choosing a paid data supplier are found in the PRSA Tactics article, Deliverable Objectives: Considerations for Creating Measurement Plansby Jackie Matthews and Pauline Draper-Watts.

  • Ask potential providers whether or not they supplement aggregated data feeds with their own web-crawler, and if so, how deep it goes into sites?
  • Does the crawler reach the home page only, or can it penetrate down to the specific URL level?
  • What kind of data cleaning will be done?
  • How much storage will one receive, how far back does it go, and how easy will it be for one to access the data?
  • Make sure to understand as much as possible about the measurement formulas and weightings used by the tool, recognizing that no formula will be perfect (and many are proprietary).
  • Finally, check references, account team, and pricing.

We’ll leave off here and pick-up next month with Target Audience Effects, the most exciting part of this overall section.

Meanwhile, if you have questions or measurement needs, I’d love to speak with you! Send me a note at angela.jeffrey@salienceinsight.com.

### (thanks to mind42 for the image)

Angela Jeffrey

Angela Jeffrey

Angela Jeffrey is Vice President Brand Manager for Advertising Benchmark Index. A recognized measurement evangelist, thought leader, writer, and speaker for PR measurement and evaluation, Ms. Jeffrey created PRtrak™, one of the first analysis tools to cover print, broadcast, and internet coverage. Most recently, she served as Strategy Director US for Salience Insight and CARMA. She is also a long-time member of the IPR Measurement Commission. angie@adbenchmark.com
Angela Jeffrey
728 Ad Block

Related posts

728 Ad Block