Ask The Experts

Is it Unethical Not to Measure?

Eight Measurement Pros Respond:
Walter Barlow, Don Bates, Fraser Likely, Tom Nicholson, Frank Ovaitt, David Rockland, Ward White and Lou Williams

The Measurement Standard thanks each of our experts for their time and effort. We would enjoy hearing from any readers who wish to provide their own responses or comment on the discussion below. Please write to The Editor.

The Question:

In the last issue of The Measurement Standard we put forth the provocative notion that it was unethical not to measure public relations results. (See these articles by Jim McNamara and by Katie Paine.) We asked the experts to respond, and here are their answers in chronological order...

The Answers:

From Lou Williams,
L.C. Williams & Associates

I wouldn't necessarily call it unethical to not measure. Silly. Dumb. Paranoid. Maybe. Most of the time, I think people truly don't know how to measure, are not willing to spend the money to measure... or just plain afraid of the results. One other thought: For those fearful of getting in over their heads, it is frightening to figure out what should be measured, not too mention how much to spend on measurement. Having said all of that, I'm sure there are people who are acting unethically when they don't measure... but, it's a bit of a broad sweep to call everyone unethical, just because they don't do it.

From Don Bates,

I agree. "Unethical" is reaching to say the least and it puts the onus on us as practitioners rather than on the organizations for which we work. Maybe "unprofessional" is a better way to say what I think Katie is trying to suggest, but only in certain organizations and under certain circumstances.

In my 12 years running my own agency, I never had a client that wanted measurement from us. They wanted ink. That's what they hired us for. That was the measurement. That's how they evaluated our success and whether they would continue paying us our fees for another year. On the other hand, my agency emphasized, as I think most agencies do, publicity. We did little or no government relations or investor relations. And only occasionally did we engage in crisis communications.

As I think about the argument, I might consider it "unethical" not to bring up measurement and evaluation in situations where doing it would be truly helpful to the public good; e.g., when the federal government spends tens of millions on drug use prevention and fails to establish evaluative criteria to determine whether the effort was worth the bucks spent and to assess what might be done differently going forward based on the effort. Certainly, we should all make the case for measurement with clients when it makes sense, but I wouldn't stake my relationship on a yes/no answer.

Thanks for speaking up. Too much of this stuff goes unchallenged and becomes the "truth" for the uninitiated.

But BIG KUDOS to Katie for saying something in a way that forces us to think from another perspective on an important issue. We don't criticize ourselves enough as PR professionals. Compared to the advertising and marketing communities, we're whimps when it comes to putting our feet to the fire.

I think it would be great if the Institute could develop some "standards" for when measurement and evaluation are important, critical or essential.

From Tom Nicholson,
Director, Public Relations and Communications at Sears

Every once in a while an editor needs to whack the hornet's nest with a stick just to see what's in there.

I like the thought of calling non-measurement unethical. After all, most of us have been entrusted with significant budgets with the clear expectation that we will invest those budgets rather than simply spend them. One reason (among many) that so many PR folks are not taken seriously and do not have a meaningful seat at the table is because we are either afraid of what measurement might tell us, or we simply don't fully understand our responsibilities. Ethics can be a very wide net, and thinking about our responsibilities to our employers in that way is not such a bad thing. Who doesn't want to do the very best they can at their job? And, how do you get better without measurement?

If a doctor can be sued for malpractice for not keeping up with the latest medical advances, maybe we should think about the possibility of PR malpractice suits. There's a thought that might continue to stir the hornet's nest.

From Ward White,

irresponsible, maybe ... but "unethical" is sort of throwing a hand grenade to kill the fly ... a rhetorical overreach that may actually end up as counterproductive because it's my guess that the salvo will (mostly) be dismissed out of hand ... and our subsequent messages can be equally dismissed as "Oh, it's just those kooks again." Last time I whacked a hornets' nest, I got stung ... honest-to-God truth.

From Tom Nicholson,
Director, Public Relations and Communications at Sears

Actually, you have a very valid point. We certainly don't want to lose credibility -- but I still like the question and the thoughts it provokes.

From Frank Ovaitt,

I have to agree with Ward. I'm a long-term proponent of research and measurement, but also a reality-based practitioner (at least I like to think that). I'm afraid the "unethical" charge is not only overreaching, but would probably be perceived as self-serving as well.

From Walter Barlow,
Research Strategies Corporation

It isn't a matter of ethics, as I see it, but rather a function of sophistication in both communication and in top management's own evaluation. I would be much more in favor of a position that forecasts the time when no communication expenditure of any size in the future will not be evaluated professionally as part of the process.

I look forward to the time when professionalism in research at the client level will be such as to provide proof that evaluation has been done. Perhaps too much of a pipe dream, but stockholders, for example, of a company should feel able to check the validity of any communications program through specially generated evaluation.

I may be assuming too much, but as evaluation gets better and particularly better understood, it will be the unusual business communicator who is not going to be able to produce figures and evaluation, if requested.

From Fraser Likely, APR,
Public relations management consultant:

It may be unethical not to measure - but it sure is unethical to measure unprofessionally!

I got stung by hornets on Sunday. I used a focus group to do some qualitative formative research (my 86-year-old mother who wanted that hornet's nest on the side of her garage down NOW).

Unfortunately, I didn't do broader quantitative research. If I had surveyed the hornets I'm sure I would have found out that there were Olympic calibre flyers amoung the nest. I would then have been able to adjust my communications program accordingly. I would have yelled "Open the door, now!" much sooner as I was running.

Final communication program evaluative measures were:

Outputs: 1 yell, 3 screams

Outtakes: Total recall and retention of the message sent by the one who landed on top of my bald head.

Outcomes: Behavioural change:

1. I couldn't get to my van for 24 hours because of the swarm of hornets by the garage.
2. When I left they were starting to rebuild a new nest.

Outgrowths: My corporate reputation remains intact with my mother. Our relationship is as strong as ever, though there is some slippage on the contributing factor of mutual trust: she says now that she had told me to use a longer pole so to be further away, and on my part, I perceived her to be agonizingly slow on opening the #*% door.

Maybe it was unethical, but I chose not to canvass the hornets - the stingholder stakeholders - though they really are a defined public now, on any changes in their perception of our likely Likely reputation.

I do feel our communication with them (they say it's symmetrical; I say it's asymmetrical) has changed, forever!

Unethically yours, and with apologies to Ward White for stealing his story line ...

From David Rockland:

The idea of saying that it is unethical to not measure the results of public relations raises a few questions:

Who is not being ethical? Is it the PR firm who doesn't propose measurement?Is it the client that decides they don't have the budget for it? Or is it the research side of public relations that has yet to effectively demonstrate that measurement can be standardized to demonstrate a return to business performance?

And, what kind of measurement is ethical and which is not? For example, is a pile of clips or number of impressions really measurement? Or, is a clear dollar return on investment what is really measurement? Or, is the answer somewhere in-between in the form of changed awareness, comprehension, attitude and behavior?

Measurement of public relations is possible. Calculating a return on investment is possible. Distinguishing the effects of public relations from other forms of marketing communications is possible. However, we rarely do them because it is not easy, there is not a standardized approach, and measurement is often where budgets are cut first.

What to do? An industry publication should pull together the right people to come up with the answers to these questions, and then publish it as "The Measurement Standard." Sign us up.