To return to the current issue's contents page, click here.

To return to the contents page of the issue that this article appeared in, click here.

Comments Please!
Send us your thoughts on this article and we will post them in our Comments section.

Jim Macnamara's
" Measuring Up"
(see below)

Transparency,
not 'Black Box' Measurement

At the Summit on Measurement last year in Portsmouth, New Hampshire, discussion ventured on to 'black box' measurement – a fascination in our industry to come up with mysterious indexes and metrics that purport to be the Holy Grail of Measurement.

Some argue that any day one of these will be the saviour of public relations – a silver bullet that shows the ROI (Return on Investment) of PR and sends management into spasms of appreciative joy as they reach for their corporate checkbooks to up their PR budget.

To quote a line from the iconic Australian movie, The Castle:"Tell 'em they're dreaming."

Open standards work for advertising, marketing and the hard sciences...
A look over our shoulder, some peripheral vision as they say, will show that in advertising and marketing, research has forged an important role through exactly the opposite. Rather than creating mysterious, near undecipherable metrics, mainstream research has relied on open transparent standards in conducting surveys, focus groups, interviews and other methodologies.

Survey researchers use tried and tested question techniques such as Likert scales and readily disclose (and are required to disclose under the codes of research societies), their sample sizes, scales, weightings (if any) and error rates. The same goes in scientific experimental research where strict documentation of procedures is required.

Does this openness lead to sameness in research and lack of differentiation? No. Open standards and conformity to standards creates confidence among clients and ability to understand the information they are presented with. Research companies compete by offering value-add in terms of interpreting data and drawing conclusions in relation to the client's objectives, rather than throwing up raw numbers.

...but PR is still looking for the secret magic formula.
Meanwhile, much of the public relations measurement confraternity seems obsessed with finding some magic formula – a Media Prominence Index, a Reputation Index based on a fixed formula, an ROI score or some other single metric that pronounces a campaign or project a success and its creator a hero.

This search for the Holy Grail in measurement is a blind alley that is confusing clients, fragmenting the industry and eroding the already sub-optimal level of confidence in public relations.

Where would standards come from?

The optimum path for public relations measurement is for bodies such as the Institute for Public Relations and the Association for the Measurement and Evaluation of Communication (AMEC) to set and actively promote basic standards for communication research.

Standards do not necessarily restrict competitiveness. For instance, standards should require, at a minimum:

• Open transparent disclosure of methodology including scales and calculations used;
• Publication of sample sizes and error rates in research.

In addition, standards setters should ideally have the courage to require:

• Avoidance of generalised invalid methods such as Advertising Value Equivalents for editorial publicity when it is known to be variable and potentially negative;
• Focus on measuring impact and effects (outcomes) rather than simply counting output data such as audience reach, opportunities to see and volume of messages.

Standards work for other industries

The computer, DVD/video and automotive industries have shown how important standards are. Until Microsoft and IBM and other computer companies got together and agreed on basic standards for operating systems, communication protocols and connectivity standards, computing was a black art beyond the understanding of most consumers, practiced by men in white coats (yes, there were no women in computing then). Standards did not prevent computer companies competing – in fact, they provided a platform and an environment for growth and competition.

Imagine the automotive industry if there were no standards in terms of how we indicate when we wish to turn, whether the ignition key is turned clockwise or anti-clockwise, where the gear shift is located, and so on. Without standards, consumers are confused and markets do not grow.

The video industry provides a good example of the havoc that lack of standards creates. The presence of PAL, NTSC and SECAM standards, and the options of three-quarter inch, half inch and quarter inch tape, contributed to its downfall and early replacement by a uniform standard of digital video disks. With CD and DVD technologies conforming to basic standards worldwide, the music and video industry have enjoyed growth and improved efficiencies.

Meanwhile Nora Jones, Jack Johnson and Andrea Bocelli can each produce their own unique products within these standards and consumers flock to buy them in confidence. It's time for the PR industry to sing its song – but in key, conforming to the rules of music and not some new invented scale that leaves our audiences scratching their heads and wondering what it all means.

Dr Jim Macnamara BA, MA, PhD is Group General Manager Research with Media Monitors – CARMA Asia Pacific following the acquisition of the CARMA Asia Pacific franchise by Australia's largest media monitoring firm. He has a 30-year career in the media and PR in Asia Pacific and is the author of 11 books on media and communication.

You know you need to measure your results, but chances are there’s never been enough money in your budget for evaluation. Until now.
KDPaine & Partners’ new Do-It-Yourself Dashboard system combines a Web-based application with professional consulting to enable PR professionals to customize their own PR dashboards. Look here for more information.

1. You’ll learn how to use hard numbers to prove the results of your PR efforts. (Plus, it's free.)

2. You’ll learn which are the right vendors for your measurement projects. (Yes, it's free.)

3. You’ll learn how to design your program right from the start to be easily measureable. (Plus, yes, it's free.)

Click here to
get your free
subscription now!

51c Durham Point Road, Durham, NH 03824
603-868-1550 fax: 603-868-3346 www.measuresofsuccess.com