Department of Controversy

The Ethics of PR Evaluation

by Jim R. Macnamara BA, MA, FPRIA, AFAMI, CPM

Summary: Diligent, effective PR evaluation is not simply a practical problem, but a fundamental issue of principle. For public relations to gain the professional status it seeks, PR practitioners need to recognise that conducting research and evaluation to the best of their ability—and doing so using rigorous, scientific methods—are ethical requirements. To recommend and commit client and employer budgets to unsubstantiated and unevaluated programs is unethical as well as unprofessional.

Evaluation has been the subject of a worldwide debate in public relations for the past decade and many practitioners feel harangued on the issue. However, to paraphrase respected academic Jim Grunig, evaluation is like sin. Even though we know it is wrong and are constantly warned about it, people continue to sin and PR practitioners continue not to do evaluation.

To attempt to elevate evaluation to an ethics level may be seen as an over-statement and missing the point on practical issues such as cost, which is most commonly raised as the reason for not undertaking evaluation. But cost is not the barrier to evaluation that it is presented to be, as demonstrated in a number of papers and models of evaluation which show that low-cost and even no-cost methods such as accessing secondary data (i.e., existing research), feedback, case studies and pre-testing can be effective in evaluating many PR activities.

The availability of no-cost, internal do-it-yourself evaluation methods and new low-cost software programs and tools that have been specifically developed for public relations, indicate that cost is an excuse that masks deeper, underlying reasons for public relations remaining largely unmeasured and unaccountable.

It is perhaps only when public relations brings evaluation into the centre of its focus and recognises it as a fundamental issue of principle, and not simply a practical problem, that the rhetoric on the topic will translate into practice.

There are ethical issues involved in evaluation at two levels: philosophical and methodological.

Philosophy of Evaluation

Philosophically, PR practitioners widely recognise evaluation as desirable. An International Public Relations Association (IPRA) study in 1994 found 76 per cent to 90 per cent of practitioners viewed evaluation as “necessary.”¹

However, the gap between awareness and behaviour remains wide. IPRA’s multi-national study found between 14 per cent and 25 percent of PR practitioners regularly undertake objective evaluation. Research usage rates of below 15 percent have also been reported in a number of other studies, including Walker (1994)² and Macnamara (1993)³.

Every day, PR practitioners make recommendations to their employers and clients proposing the expenditure of hundreds of thousands or sometimes millions of dollars, and suggesting changes to the strategic direction of companies and organisations, with little or no research basis in some cases. If their recommendations are accepted, which is itself an area for discussion about research, they implement programs with only scant tracking and measurement to determine whether the strategies adopted are effective.

Is that responsible? Is that moral? Is that ethical?

Throughout management and the professions, the inter-related issues of accountability, governance and due diligence have become recognised and accepted as core principles which must underpin all management thinking and action.

PR has apparently not accepted and adopted these core principles, judging by its willingness to propose and implement programs involving significant expenditure and often major policy and strategic initiatives without:

1. Rigorous investigation and sound research data to indicate that the proposals are justified and appropriate, and
   
   
2. Rigorous evaluation to ensure objectives are being achieved.

In the private sector, accountability, governance and due diligence in all activities are required by shareholders and, increasingly, by the community. In the public sector, PR practitioners carry even wider responsibilities, as they are spending taxpayer’s money.

To glibly commit either shareholders’ or public funds to programs and campaigns based only on intuition and personal experience, without an objectively researched basis to recommendations, and rigorous evaluation of the effectiveness of expenditure and activities, has to be viewed as highly questionable from an ethical standpoint.

Methodology of Evaluation

The second level of ethical concern in relation to evaluation concerns methodology.

Evaluation can be undertaken using informal as well as formal methods. Informal research includes general feedback and interviews that are conducted casually and not within a statistically selected sample group, for instance, and has less stringent requirements for accuracy than formal research. However, all formal research must be undertaken in a systematic way and diligent effort should be applied to ensure that findings are reliable.

Presenting misleading or erroneous results intentionally is clearly unethical. However, doing so through ignorance and failure to consult readily available literature or seek advice is also ethically questionable.

There is extensive evidence and advice available to PR practitioners, for instance, that Advertising Value Equivalents are invalid. The Public Relations Research and Evaluation Toolkit produced by the Institute of Public Relations (IPR) in the UK, a Research and Public Relations Position Paper published by the Public Relations Institute of Australia, and a number of books and articles have pointed out the major shortcomings and potential misrepresentation of AVEs. Yet PR practitioners continue to use them extensively.

Equating editorial publicity to the value of equivalent space purchased at advertising rates is a measurement method that promotes and encourages volume rather than quality. It also ignores the obvious positioning, layout and content differences between editorial and advertising.

Editorial may have more impact than advertising in some circumstances. In other cases, editorial may contain criticism, it may favourably compare competitors, it may be poorly positioned and displayed, and it may even be in media that do not reach the client’s or employer’s target audience.

Can a PR practitioner ethically present such editorial and claim that it has the same value as expensive advertising, the content of which is always favourable, which is designed to maximise impact, and which is placed only in strategically important media?

AVEs allow PR practitioners to, intentionally or unintentionally, mass-distribute publicity widely in easy target and often non-strategic media and then claim that their outputs are worth many hundreds of thousands or even millions of dollars when clearly, in some cases, they are not.

In extreme cases, PR practitioners multiply the equivalent advertising value by a ‘credibility factor’ ranging from three to nine, or even more in some cases, on a popularly-canvassed assumption that editorial is more believable than advertising. Respected US academic, Jim Grunig, warns that there is no research basis for credibility multiples and no sound foundation for even a 1:1 comparison with advertising. Such calculations are spurious and warnings have been circulated in PR journals for several years.

The Institute for Public Relations in the US in a booklet, Guidelines and Standards for Measuring and Evaluating PR Effectiveness says of such ‘credibility multipliers’:

Some organizations artificially multiply the estimated value of a possible editorial placement in comparison to advertising by a factor of 2, 3, 5, 8 or whatever other inflated number they might wish to come up with, to take into account their own perception that editorial space is always of more value than is advertising space. Most reputable researchers view such arbitrary ‘weighting’ schemes aimed at enhancing the alleged value of editorial coverage as unethical, dishonest, and not at all supported by the research literature. ⁴

Basic positive/negative/neutral ratings of media publicity are also questionable as an evaluation methodology because they are simplistic, subjective and do not take into account communication of key messages or reach of target audiences and, therefore, are unrelated to achievement of objectives.

Pressure from clients and employers is cited in defense of invalid and simplistic PR evaluation methodologies. “My client wants it,” or “My boss likes to see dollar values,” are common justifications for AVEs and other such evaluation shortcuts.

However, is it acceptable for accountants to falsify tax returns because their clients exert pressure to reduce their taxation? Or for lawyers to arrange for witnesses to lie in court because their clients pressure them to prove their innocence? Such behaviour from professionals is widely viewed as unacceptable.

Public relations needs to set and meet standards that are no less rigorous if it is to be recognised as a profession and gain credibility and respect.

Evaluation is not undertaken in public relations for a number of reasons, the least of which may be cost.

• Most PR practitioners do not have research skills or the knowledge to plan and undertake research. Research skills and knowledge need to be developed urgently, or partnerships formed by PR practitioners with research professionals in the same way as they have done with graphic designers, printers and other specialists.
  
  
• Objectives set for PR are often broad and unmeasurable. Objectives need to be specific, relevant and realistic.
  
  
• Accountability and responsibility are not seen as ethical issues at the core of public relations practice. Keeping the client or boss happy is often seen as enough. While the latter is important, broader long-term achievement of objectives, effectiveness and efficiency need to become values accepted and applied in public relations.

In Conclusion

For public relations to become a true profession and gain the status it seeks, PR practitioners need to recognise that conducting research and evaluation to the best of their ability—and doing so using rigorous, scientific methods—are ethical requirements. Conversely, as harsh as it sounds, it needs to be recognised that continuing to recommend and commit client and employer budgets to unsubstantiated and unevaluated programs is unethical as well as unprofessional.

Jim Macnamara is a prominent international authority on measurement and evaluation of public relations and corporate communication. After more than 20 years working in the media and public relations, he founded and is CEO of The MASS Communication Group, an independent communication research and consulting group, and operates the Asia Pacific franchise of global media analysis firm, CARMA International.

References

Michael Fairchild (researcher and Ed), “The Public Relations Research & Evaluation Toolkit,” Institute of Public Relations (IPR) and Public Relations Consultants Association (PRCA), UK, 1999.

“Research and Evaluation,” Position Paper, Public Relations Institute of Australia, Sydney, 1998.

Jim Macnamara, “The Ad Value of PR,” Strategic Communication Measurement, Melcrum, November, 2000.

Footnotes

1. “Public Relations Evaluation: Professional Accountability,” International Public Relations Association (IPRA) Gold Paper No. 11, 1994.

2. Gael Walker, “Communicating Public Relations Research,” Journal of Public Relations Research 6 (3), 1994.

3. Jim Macnamara, “Public Relations & The Media: A New Influence in Agenda-Setting and Content,” MA thesis, Deakin University, 1993.

4. “Guidelines and Standards for Measuring and Evaluating PR Effectiveness,” Institute for Public Relations, Florida, USA, 2000.

New! A State-of-the Art
PR Dashboard for $50 a Month
You know you need to measure your results, but chances are there’s never been enough money in your budget for evaluation. Until now.
KDPaine & Partners’ new Do-It-Yourself Dashboard system combines a Web-based application with professional consulting to enable PR professionals to customize their own PR dashboards. Look here for more information.

1. You’ll learn how to use hard numbers to prove the results of your PR efforts.

2. You’ll learn which are the right vendors for your measurement projects.

3. You’ll learn how to design your program right from the start to be easily measureable.

Click here to
subscribe now!
(It’s 100%
money-back guaranteed!)

51 Durham Point Road, Durham, NH 03824
603-868-1550 fax: 603-868-3346 www.measuresofsuccess.com