Advancing communications measurement and evaluation

Katie Paine’s 13 Favorite Papers From IPRRC 2012


PR research types get a little time in by the pool at IPRRC 2012.

The best of the 15th Annual International Public Relations Research Conference.

 by Katie Paine, CEO, KDPaine & Partners

The International Public Relations Research Conference (IPRRC) is held each year in March in Miami. The consensus on this year’s IPRRC is that the quality of discussions and papers has improved significantly. Whether, as some suggested, it was due to more rigor on the part of the reviewers (transparency alert: I am one of those reviewers) or just that the reputation of the conference is attracting higher level papers, makes no difference. The conversations were stellar. Social media was definitely the subject du jour, with a whopping 32% of the 108 papers presented involved social or digital media.

See this Flickr set for lots and lots of photos of the conference.

Click here to download the (12 MB) IPRRC15 Program

— Thanks to Constantin Basturea for putting together this Storify recap of the conference. It's got photos, tweets, videos, and links to resources and blog posts.

Here are my 13 favorite papers:

The Hot Potato Award

Matt Ragas, DePaul University, Matt Brusch, National Investor Relations Institute, and Alex Laskin, Quinnipiac University, for What Matters? An Exploratory Study of the Criteria Investor Relations Professionals Use to Measure the Success of Investor Relations Programs, their in-depth examination of the conflicted relationship that investor relations has with measurement. They conducted what might be the first-ever measurement survey of National Investor Relations Institute members. The truly astounding result was that 87% feel that share price should not be used as a valid measure of success of an IR program, but there is no agreement on what might be used as an alternative. There is strong belief in the value of building relationships with the investment community, but no one is measuring that. Most rely on feedback from the C-suite as a measure of success.

Runner Up:
Marcia Watson DiStaso
, Penn State, and Marcus Messner, Virginia Commonwealth University for taking on Wikipedia and the five top banks with Cite that Crisis: An Analysis of the References in Wikipedia Entries of Major Banking Institutions. They found that overall the number of news references cited in Wikipedia entries for the banks increased dramatically in the last two years, as did the percentage of negative references. In fact, percentage of negative references in 2011 exceeded even 2009. The top positive reference cited was Bloomberg, while the reference for most negative citations was the New York Times.  Not surprisingly, Goldman Sachs, Citigroup, and Bank of America had the most negative citations.

Most Likely to Be Cited

Tina McCorkindale, Appalachian State University, Marcia Watson DiStaso, Pennsylvania State and Hilary Fussel Sissco, Quinnipiac University. With two of the hottest search terms in its title – “Facebook” and “millennials” – their study, How Millennials are Interactive with Organizations on Facebook, is bound to become a classic. Based on a survey of 18-29 year-olds on three university campuses, it tells us a lot about that much-desired demographic. But, lest you think it’s all just hype, the findings are illuminating:

  • 75% of millenials have “liked” an organization
  • 45% have “liked” products
  • 48% have accepted friend requests from organizations (but before all you brand managers get too excited, most of the organizations are non-profits)
  • 44% have donated supported or joined a cause
  • 86% access Facebook every day
  • 57% access Facebook via a cell phone. And what are they doing during the hours they spend there? Mostly checking out their friends' profiles, updating their own statuses, checking messages, and staying in touch.

In other words, brands and causes get short shrift in the overall scheme of things. Brands beware, there are few that break through in the clutter that is now Facebook. Coke, Target, Nike, Taco Bell, and Victoria’s Secret were most popular, but millennials are far more likely to “like” sororities and fraternities, sports teams, and non-profits.  Only 10% accept friend requests from organizations. But 18% accept friend requests from people they don’t know. When they do go to organizational sites, they do it for discounts and free samples, especially those they can pass on to friends. 43% said they learned about fan pages because friends were fans. They also aren’t terribly loyal: 42% said they have left or “unliked” a page, mostly because the organization was mostly selling products. Other reasons cited were “annoying or excessive notifications, boring content, lack of dialog or they were no longer interested in the content.” 

Runner Up, (and Special Most Hilarious Trick Survey Question Award):
Donald K. Wright, Boston University and Michelle Drifka Hinson, University of Florida for their longitudinal analysis of the use of social media in PR: A Four-Year Longitudinal Analysis Measuring Social and Emerging Media Use in Public Relations Practice. The study continues to demonstrate the extent to which social media has changed the fundamental nature of PR. More than a third of respondents, up from 26% in 2011, now spend more than a quarter of their average workday in new media. 15% say they spend more than half their working time in social media. And, it continues to appear that Corporate Comms and PR are winning the debate about who should own social media, with the vast majority saying that those departments are managing it.

The good news is that 57% of respondents report that they are measuring what is being said about them in social media, up from 52% last year. Unfortunately, only 31% measure the outcome or impact. Facebook and Linked In were voted most important of the social media networks, despite the fact that a significant number of respondents felt that Search Engine Marketing and blogs should be more important. Our award for Most Hilarious Survey Trick Question goes to a test of the integrity of the responses. The authors inserted two completely fictional social networks into the list of possible platforms. The good news was that the majority of people said that they did not use them—but one does have to wonder about the three percent who reported using them frequently.

Sanest Approach to a Controversial Topic

Fraser Likely, Principles for the Use of Return on Investment (ROI) Benefit Cost Ratio (BCR) and Cost Effectiveness Analysis (CEA) Financial Metrics in a Public Relations/Communications Department. Given the brouhaha currently underway after a SXSW panel on ROI blew up, this paper was amazingly timely. Fraser suggests that there are other, perhaps more appropriate, financial measures that PR people should use rather than ROI. He argues that these other metrics are more useful in making decisions about what paths or strategies to choose. His specific Principles to Follow When Attempting to Employ Financial Measures with a PR/C Program or Campaign:

ROI Metric

  1. The PR/C campaign must establish baseline behaviors before it sets objectives for incremental behavioral change.
  2. The PR/C campaign must track its targeted audience through all stages of cognitive, attitudinal and conative or behavioral change.
  3. The subsequent function charged with converting the foot-in-the-door into a sale, signature or some other act must track these same people through the stages of the conversion process.
  4. A PR/C campaign overlapping more than one fiscal year needs to look at the investment that will be risked with the new program and examine hurdle rates, opportunity costs, and diminishing returns.
  5. The financial return is the financial value for each closed sale, each closed signature, or each final action.
  6. With any ROI calculation there is no estimating of PR/C intangible outcomes and then trying to give them a financial value.

BCR Metric                                                 .

  1. The projected campaign must have a clear path to its expected end state.
  2. All potential investments must be accounted for, including PR/C's and a subsequent function's investments.
  3. The financial value for each possible closed sale, each closed signature, or each final action is required.
  4. When utilizing compensating variation to assign monetary benefits, accuracy in the identification of campaign effects and accuracy in the valuation of those effects are necessary.
  5. A positive benefit cost ratio means that expected financial returns will exceed financial costs. If not, the program needs to be altered.
  6. Proposed programs can be examined based on a comparison of benefit cost ratios. At the same time, a baseline ratio should be established first. The baseline is the minimum ratio acceptable for the introduction of any and all new programs or campaigns.

Cost Effectiveness Metric

  1. All potential investments must be accounted for.
  2. An effectiveness benefit must be identified, one that is universal to ensure a solid comparison.
  3. The effectiveness benefit must be quantifiable, and be quantifiable across the comparison. Then the ratio of cost to effectiveness benefit may be determined and compared across all contenders.

His bottom-line:

  1. PR & Communications needs to use common financial metrics like ROI, BCR, and CEA as they were designed.
  2. The PR/C function can only measure ROI of communication campaigns if that measurement is part of a larger program and if that program ROI measurement exercise includes all attributable costs from across the organization and a net financial return at the level of the organization.
  3. Compared with ROI measures, BCR and CEA financial metrics have the possibility of greater utility in providing valuable financial measures.

Most Thought Provoking:

Minjeong Kang's Understanding Public Engagement: Conceptualization and Measurement definitely presented the most intriguing new idea of the conference, and probably of the year. Kang ventured into the thorny world of measuring social media engagement from a completely different perspective than most of us have. She looked to the fundamentals of relationship theory and presented a proposed model for measuring engagement based on them. Her thesis is that you need to understand how people really feel, rather than just what they click. She suggests that engagement is characterized by affective commitment (emotional bonding and pride that brings additional efforts to the organization), positive affectivity (elevated emotional tone of the engagement state), and empowerment (conceptualized as self-efficacy and impact that results in motivated behaviors). She investigated her theory with a survey of 4,425 randomly selected current patrons of a professional theatre organization in the United States. The results showed that engagement is all of those things and has a positive impact on results. Now the only question is: What are we going to do with all that Google Analytics and Facebook Insights data ?

Runner Up:
Koichi Yamamura, Media Gain; Junichiro Miyabe, Hokkaido University; Naoya Ito, Hokkaido University; and Hitoshi Wada, Tokyo International University,  for their paper Quake Hits PR: The impact of 3.1 earthquake on public relations in Japan. They studied the Fukkushima crisis from the perspective of its impact on PR in Japan. Not surprisingly, the most frequent response on the part of organizations was to increase corporate giving. However, the other impacts were more intriguing:

  • 67.5% of organizations surveyed said they established an emergency headquarters,
  • 82% issued a special message from the CEO, and
  • 58% refrained from issuing press releases on their CSR activities. 

The "Most Disturbing to Bloggers" Award

Is Ghost Blogging Like Speechwriting? A Survey of Practitioners About the Ethics of Ghost Blogging, by Tiffany Derville Gallicano, University of Oregon, Kevin Brett, Brett Communications, and Toby Hopp, University of Oregon found that most PR people surveyed (71.1%) agreed or strongly agreed that “It’s okay for an organization to have blogs that list executives as authors, even though they are really written by someone else, as long as the ideas come from the executives and they approve the message.” Another 45% said that they agreed that “It’s okay for an organization to not disclose the agency's assistance with writing blog posts under a client’s name.” For those of us who have been living in the social media world for a long time, this goes against the transparency and authenticity that has been so appealing from the beginning. I fear for the credibility and trustworthiness of bloggers everywhere.  

Most Interesting Studies of Crisis Theory in Action

As always, many of the papers analyzed corporate response to a crisis. Many focused on BP, but we’ll talk about them later. Not surprising, the god and goddess of crisis theory, W. Timothy Coombs and Sherry J. Holladay from the University of Central Florida had me most intrigued. For Blanket Verses Qualified Denials: Exploring Variations in Denial as a Crisis Response Strategy they conducted a controlled experiment to investigate two different responses to a crisis: If you are going to deny culpability, is it better to hedge your bets and be open to investigation, or to completely deny everything? As it turns out, either way, if you are later found to have any association with the crisis, then the very fact of your denial increases the anger and worsens your reputation. In other words, while denial can help to separate an organization from the dangers associated with a crisis, the damage is magnified if the organization is later found to be responsible. So, all you lawyers out there, listen up: If we follow your advice and deny everything, the damage to the organization’s reputation may well outweigh any potential legal liabilities.

In Corporate Social Responsibility as a Image Repair Strategy: A Case Study of BP’s Response to the Deepwater Horizon Crisis, Denise Sevic Bortreee and Breanna Thorpe from Penn State conducted an exhaustive analysis of the BP Oil Spill crisis focusing on BP’s use of video. Some 315 videos were posted by BP on its YouTube channel between 4/2010 and 10/22/11. Turns out that the videos promoting its own corporate social responsibility were the most disliked. The study also found that BP’s videos that featured people described as “Gulf Coast residents” but who were mostly male Caucasians evoked the worst response. There was some speculation in the subsequent discussion that the plethora of videos may have been generated by BP’s PR agency just to make sure that when you searched for BP Oil Spill, the first video that you saw wasn’t the dramatic real-time video of oil gushing from the ocean floor.

Miscellaneous Interesting Factoids:

Developing Online Newsroom Evaluation Index, is an intriguing state-by-state study of the use of social media by tourism departments by Kyung-Hyan Yoo at William Paterson University and Jangyul Robert Kim of Colorado State. As it turns out, states vary widely in their adoption of on-line newsrooms, with Colorado, Delaware, Virginia, and South Dakota using them most effectively. Georgia and Utah use them not at all. Social media use was also studied, with North Dakota taking the most advantage of social media tools and Rhode Island coming in dead last. New Hampshire, by the way, ranks seventh in the effectiveness of its online newsroom but  37th in use of social media.

Studies show that when stakeholders understand and are aware of corporations' CSR activities, they respond positively. But they need to know about those activities, so Using Social Media to Report Corporate Philanthropy by Hen Ping Lee and Sherry Holladay was fascinating. Just a bit more than half the companies studied provided links to social media. Half of the posts were through Twitter, primarily bragging about cash donations. Researchers found that while the information was there, it was hard to find and difficult to access.

In Analyzing BP’s Twitter Response to the Deep Water Horizon Accident Using Coombs SCCT, Laura Walton, Skye Cooley, and John Nicholson of Mississippi State University also took on the BP oil crisis with their examination of how BP used Twitter to respond to the crisis. As it turns out, and contrary to most expectations, only 35% of BP’s tweets attempted to diminish the crisis. The majority (56.7%) attempted to ingratiate themselves by expressing regret, apology, concern, and compassion. In terms of spokespeople, the federal government was, in fact, the most frequent commenter, due no doubt to the fact that two months into the crisis, the feds took over the Twitter account. 


Katie Delahaye Paine is CEO of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements. Katie and Beth Kanter are authors of the book “Measuring the Networked Nonprofit,” to be published this year by Wiley.


Bill Paarlberg
Visit me on

Bill Paarlberg

Bill Paarlberg co-founded The Measurement Standard in 2002 and was its editor until July 2017. He also edits The Measurement Advisor newsletter. He is editor of the award-winning "Measuring the Networked Nonprofit" by Beth Kanter and Katie Paine, and editor of two other books on measurement by Katie Paine, "Measure What Matters" and "Measuring Public Relationships." Visit Bill Paarlberg's page on LinkedIn.
Bill Paarlberg
Visit me on
728 Ad Block

Related posts

728 Ad Block