Shopping for an Agency that Measures Up?
You should be.
We review and rate seven top agencies to find out how and what they measure.
Fleishman-Hillard, GCI Group, Golin/Harris, Hill & Knowlton, Ketchum,Manning Selvage & Lee, and Text 100.

By: Lindsey Wyckoff, Staff Writer

To complement our other coverage of agencies and measurement this month, we decided to take a look at how some of the top PR agencies measure results for their clients. We know we are not the only ones out there tooting the measurement horn, so we put some agencies to the test, in hopes of discovering who is equally excited about measurement.

Here are some questions we tried to find answers for, and that you should be concerned with when you consider an agency’s measurement practices:

• Do they offer some sort of measurement system to clients?
• What methodology do they use?
• How much do they recommend their clients spend on measurement?
• Do they do their own research?
• Do they have a measurement expert on their staff?

The first thing we discovered about PR agencies and measurement is that, for the most part, there is not a lot to learn from their Web sites. The amount of jargon and PR-speak was overwhelming, and it seemed as though many agencies attempted to offer as little information as possible. So, if you want to learn more about an agency, especially about their measurement practices, we highly recommend you contact an actual human.

After speaking to numerous humans ourselves, the conclusion emerged that while they were rather helpful, measurement in the agency world is a boring topic. No fault of the agencies, of course, but as Mark Rozeen of Golin/Harris (see below) so nicely puts it, “In the Mexican restaurant of PR, we’re all selling enchiladas.” The difference between the gourmet and the generic comes primarily in the attitude and the approach.

The attitude isn’t exactly what one would expect. Regardless of the question, the most common responses were “varies by client,” “depends on the client” and “whatever the client prefers.” Which naturally makes it difficult for a prospective client to find out anything. Rather droll, don’t you think?

Now, the nuances of the client/agency relationship are complicated at best, but the success of measurement requires both sides be vested in the program. One agency even stated that they are unwilling to make recommendations as to their clients’ spending. If that doesn’t put a damper on measurement right from the start, we don’t know what does. As we here at The Measurement Standard regularly remind readers, the importance of budgeting and planning for measurement from the get-go cannot be stressed enough. It should be a part of the PR program as a whole, not an afterthought or an addition. This means setting an appropriate budget with the client up front. It is generally agreed that research and measurement should account for at least 10% of the PR budget.

For all the cookie-cutter answers we received, most agencies did manage some originality in their responses concerning Ad Value Equivalency. We all know how controversial AVEs are (see Jim Macnamara's article elsewhere in this issue), and that even in the most optimistic of lights, they are considered of questionable validity. While the majority of agencies recognize this fact, they also realize that AVEs can be useful in sort of proving the value of PR to certain number crunching clients. So, agencies straddle the line here: They try to avoid AVEs, but if a client requests them, they are more than willing to provide the client with what they want. Of the agencies we spoke with, only Fleishman-Hillard stated that they do not use AVEs.

Fleishman-Hillard
Fleishman-Hillard includes a comprehensive explanation of their measurement services and methodolgy on their Web site. Under the brand name e.c.h.o (every communication has objectives-not to be confused with Echo Research, Sandra MacLeod’s UK media analysis firm), Fleishman-Hillard Research offers competitive analysis, communication audit, message testing and media content analysis services as well as benchmarking surveys and publicity polls. Before beginning any program, they determine initial attitudes and behavior against which to benchmark results. They do not use weightings to evaluate results. Clients choose which measurement methods will be utilized before their program is implemented and the prices are factored in as part of their service offerings. They recommend that clients spend at least 5%-10% of their budget on measurement. Their measurement staff is 20 people, with a number of experts available. They perform the majority of their research in house, but may utilize outside sources for data collection and occasionally tabulation.

GCI Group
At GCI Group, measurement is incorporated into and individualized for each client’s program. They use Ad Value Equivalency when the client calls for it, mostly for larger clients who spend a lot on advertising. GCI often performs research in house, depending on the client’s preference and the size of the project. Teams of 3-5 people work on each project. They do not make recommendations as to their clients’ spending on measurement.

Golin/Harris
Golin/Harris offers their proprietary Performance Barometer Scorecard to clients. Survey results are turned into a scorecard number used to plot clients against themselves or other brands. Their measurement work often occurs in conjunction with NFO Worldgroup and Insight Express. At Golin/Harris evaluation is used to show performance and benefits consistent with other lines of business in the company. And they do research up front to determine client metrics. Mark Rozeen is their measurement expert.

Hill & Knowlton
Hill & Knowlton offers Radar (their own measurement system) to clients at no additional cost. It automates a lot of the measurement process. They perform measurement in house on a small scale, and out-source larger measurement projects to Millward Brown. (See our discussion of their services as part of this article in an earlier issue.) They keep their methodology under wraps and rework it on a case-by-case basis. Measurement is built into project costs, and there is no extra charge per se. The client decides what level of information they require and then metrics for the project are established. Hill & Knowlton do small amounts of research internally (the Washington, D.C. office supports three on-staff researchers, other staff would be added as needed). Their measurement expert is Barbara Coons; she is involved in all measurement projects.

Ketchum
Ketchum, winner of PR Week’s 2002 Agency of the Year Award, was once the home of Dr. Walter Lindemann, measurement expert and IPR Measurement and Evaluation Commission founder. With alumni like that, it is a given that Ketchum offers top-notch measurement to clients. In fact, measurement is part of their plans right from the start (check out the Ketchum Planning Process). Their research department is immense, with 1,300 employees worldwide, so they perform the majority of research on their own. For clients who prefer to do their own research, Ketchum offers measurement products like BrandBuilder, which measures awareness and behavior in the marketplace. For media measurement, they use VMS (see “Broadcast Monitoring Services” in our August 2002 issue) as a media monitoring tool in conjunction with MediaFocus and Ketchum Publicity Tracking System to measure and analyze media coverage. Other proprietary tools include BrandBuilder and the Ketchum Relationship Index. Their recommendations for client spending on measurement are made on a case by case basis. Although they try to stay away from weightings such as Ad Value Equivalency, they find that sometimes they are useful in measuring awareness. (A claim we here at The Measurement Standard find of dubious validity.) They have several measurement experts on staff, including Robyn Massey and Ask the Experts contributor David Rockland.

Manning Selvage & Lee
Manning Selvage & Lee uses its sister company, i to i research, to perform measurement. Run by measurement expert Claire Spencer, i to i research uses the i to i tracker model. Fieldwork is outsourced and i to i performs the evaluation. They recommend that clients spend 10% of their budget on measurement.

Text 100
The Global PR Measurement Survey on their Web site is the first indication that measurement plays an integral part in the PR practices of Text 100. We strongly encourage everyone interested in measurement to take the survey and sign up to receive a copy of the results. Text 100 emphasizes the importance of measurement from the beginning; it is included as a part of each client’s plan. They offer smartPR benchmark tools as well as customized measurement. The scope of the research determines whether they will perform it in house or use Biz360, but they always perform evaluation themselves. The methodology is included with the client plan, and tailored for each program. Their spending recommendations vary depending on the needs of the client. Rather than relying on a standard measure, they evaluate using quality as well as quantity. So if a story appears on the front page of The New York Times, it receives more weight than it would in a smaller newspaper. Resident measurement expert Matt Ravden serves as Director of Practices, Products and Services.

How The Leading Agencies Measure Up
After reviewing the answers that all seven agencies gave to our queries, we ranked them on a set of criteria loosely based on the IPR's Guidelines for Measuring Public Relations. The criteria included whether they used AVEs, whether they used outside independent resources, pricing/budget recommendations and general approach to measurement.

*****Five Star Rating = definitely recommended to any professional communicator who wants accurate measures of success

**Two Star Rating = acceptable approach but they don’t adhere to the IPR standards

ø = Not Recommended

New! A State-of-the Art
PR Dashboard for $50 a Month
You know you need to measure your results, but chances are there’s never been enough money in your budget for evaluation. Until now.
KDPaine & Partners’ new Do-It-Yourself Dashboard system combines a Web-based application with professional consulting to enable PR professionals to customize their own PR dashboards. Look here for more information.

1. You’ll learn how to use hard numbers to prove the results of your PR efforts.

2. You’ll learn which are the right vendors for your measurement projects.

3. You’ll learn how to design your program right from the start to be easily measureable.

Click here to
subscribe now!
(It’s 100%
money-back guaranteed!)

51 Durham Point Road, Durham, NH 03824
603-868-1550 fax: 603-868-3346 www.measuresofsuccess.com