Measuring Process
or,
How many managers does it take
to put out a press release?

by Katharine Delahaye Paine

Measure process? A real life story... Early in my career, I discovered a paper by someone at Motorola that tried to apply TQM measures to public relations. They had calculated the number of errors that could possibly be made in a press release, figured out how many they actually made and set a goal of lowering the number of errors made.

I was amazed that anyone would spend that much time and effort measuring something so unrelated to outcomes. Why bother measuring press releases when you could measure their effectiveness? Like, what did the editors actually do with them? Did they run anything? Did they communicate any key messages? Did they have any impact?

Ever since then, I’ve been telling people to measure their outtakes and impact. Now, however, I'm beginning to realize how important it is to have some sort of analysis tools to help managers figure out what process is working or not working in their departments.

Measure Process? Another real life story... One of my very first uses of measurement was to better manage the department while I was Director of Corporate Communications for a very large software company. We had just completed the launch of a major product and, unfortunately, a significant percentage of the articles that were written about the new product failed to communicate our key messages and, in fact, completely mispositioned the product.

My immediate response was to blame the strategy of launching via a major “bash.” Over the years I’d learned that free shrimp seldom makes for clear message communication. I therefore analyzed the launch coverage in some detail to figure out which publications and which reporters had “gotten” the message and whether they were at our event. The data, however, revealed no relationship: Reporters who had attended our event were just as likely to have communicated our key messages as the ones who had stayed home and read the press kit.

But, another interesting statistic popped out: Almost all the mispositioning of the product occurred in a certain segment of the trade media—those publications specifically covering large mainframe computer issues. Turns out that we had just reorganized the department and assigned responsibility for those publications to a young but talented PR manager.

Just to confirm my suspicions, I interviewed a few reporters to find out what their experience had been in dealing with my department. (Hardly the formal “communications audit” I’d recommend now, but the best tool I could muster at the time.)

As it turned out, the reporters had had difficulty getting their phone calls returned. Surprised, I took my concerns to the offending manager, only to find out that she felt that her new assignment was less prestigious than her old one, that I had demoted her and would probably fire her soon enough anyway, so she had opted to spend her time planning her upcoming wedding. After I explained that this particular group of editors was vital to the company’s future, and accurate messaging in this audience was a key measure of her success, her behavior quickly changed. And our coverage improved.

Measuring process today... 15 years later, our industry has gotten a lot more sophisticated about measuring process. Companies like GE and Caterpillar that have embraced six sigma methods for improving productivity are comparing six sigma measures to media outcomes to correlate the best activities with the best results. (See this article in Delahaye Medialink’s Gauge newsletter for more on GE’s use of six sigma methods to manage PR.)

Two of the few current authorities on contemporary process measurement are Professors Jim and Larissa Grunig of the Department of Communications, University of Maryland. In March of last year, they undertook a massive study of Los Alamos Labs (Guidelines for Formative and Evaluative Research in Public Affairs, The Institute for PR) to define nine ways to measure process, including:

1. Observations
2. Advisory groups
3. Interviews
4. Focus groups
5. Questionnaires and survey research
6. Content analyis of media
7. Cyber analysis
8. Naturally occurring information
9. Databases

The Grunigs suggest that there are other and perhaps better ways to measure the strength of your relationships with your publics—which is, arguably, what your PR program is designed to do in the first place.

Those measures are:

• Access – To what extent do members of the publics or activist leaders provide access to your PR people? What is your department’s share of access vs. another department’s or a competitor’s?
  
  
• Sharing of tasks – To what degree is your staff included in projects; to what extent do people, departments or communities involve you in their projects and share knowledge or resources with you? You might want to measure what percentage of projects your department participates in vs. what percent your department members have a leadership role in.
  
  
• Disclosure or openness – To what degree are people open and frank with each other? A good measure of this would be to count incoming media queries, complaints, suggestions, inquiries, etc. You could also conduct a reporter audit to determine their attitudes towards your staff.
  
  
• Networking – To what extent are you included in industry or community groups?
  
  
• Conflict resolution – To what extent are conflicts resolved early in the process? To what extent are they litigated?
  
  
• Credibility – To what extent is your staff or department sought out for its expertise?

The same basic PR performance measures come into play when you are measuring process – Measures of Impact, Outputs and Outcomes.

Measures of Impact

1. Did the press write what you wanted them to write, at the right time, in the right amount? In that case, the data would come from a detailed media content analysis.
   
   
2. Did your messages reach the desired audience? Did your audience retain them? Typically these would be measured with a survey.
   
   
3. Did your audiences change their behavior within the time frame? If so, you need to be tracking their behaviors, be it attendance, purchase, or votes.
   
   
4. Did your audiences or constituencies change what they perceived or believed about you? To establish what their perceptions are you will need to conduct a survey.
   
   
5. Did you establish the desired kinds of relationships? The strength of a relationship is measured both in the media, by how someone talks about you, or in a survey.

Measures of Outputs

1. How long does it take to get a press release out the door? Start measuring from the time you get the assignment, not from when you start actually writing. Track the number of days for approvals and revisions.
   
   
2. What is the optimum size of staff for the desired outcome? For example, when we measured Apple Computer’s results, I noticed that the coverage doubled in May and continued at that level for the rest of the year. I couldn’t explain the increase, so I asked the client, who said, “Oh, they doubled the staff in February.” Clearly, if you want a certain level of coverage, you need to have the staff on hand to get it.
   
   
3. How accurate are the tools you use? What percentage of your press release distribution list actually covers you on a regular basis? What percentage are still alive? How frequently does your list need to be updated?
   
   
4. What are the optimum ways to get your news distributed?

There are a number of distribution services out there all offering lower prices and better tracking than the big boys, PR Newswire and Business Wire. Do you really need all the distribution outlets offered by the bigger firms, or can you make do with some of the lower cost services?

Do the distribution services you use provide tracking? How accurate is it? Does it include online coverage?

Measures of Outcomes

According to Grunig, there are four relationship outcomes from a PR program:

1. Control mutuality – The degree to which parties agree on who has the rightful power to influence each other.
   
   
2. Trust - The extent to which people believe what you say.
   
   
3. Commitment – The extent to which one party believes in the other’s commitment to the relationship.
   
   
4. Satisfaction - The extent to which one party feels favorably toward the other. In other words: Do the benefits of the relationship outweigh the cost? Is this relationship worth spending energy on?

Each of the relationships you have with your various publics—customers, investors, employees, community, etc.—needs to be evaluated in terms of these four criteria. At least once a year you should examine those relationships and see how well they are being nurtured and sustained.

Then, when you are forced to slash your budget and are looking hard at headcount, you’ll be able to argue effectively what it takes to maintain the level of relationships necessary to achieve your business goals.

For more information on measuring process see this paper by Jim Grunig on the IPR Web site.