Advancing communications measurement and evaluation

The Measurement Life: An Interview with Jim Macnamara

Jim Macnamara speaking

Jim Macnamara has been around the public relations measurement and evaluation block a few times. In fact, Professor Jim Macnamara, Ph.D, FAMI, CPM, FPRIA, FAMEC, has been a journalist, a PR pro, owned a measurement business, written at least a dozen books on measurement, and is now Professor of Public Communication and Associate Dean at the University of Technology, Sydney, Australia.

He’s a big thinker, and to sit down and talk with him about media and communication is to get the really broad picture of how humans like to interact, and how they use technology to do so. He’s a regular contributor to The Measurement Standard, see his recent series of articles on qualitative research here, here, and hereWe caught up with him in the midst of five weeks of interviews in the U.S. and U.K. for his latest research project…

The Measurement Standard: Thank you, Jim, for agreeing to our Measurement Life interview. Question #1: What’s on your iPod, turntable or Pandora channel right now?

Jim Macnamara: Classical music for relaxing—Pachelbel, Dvorak, Brahams, etc.—and photos of my grandkids.

TMS: How did you become interested in public relations measurement and evaluation?

JM: One day around 1990, the biggest client of my PR agency (no less than Microsoft Asia Pacific) asked me to prepare a report on what we had achieved for them in the previous year. I wrote what I thought was a great report and took it to the CEO feeling fairly pleased with myself. He glanced through the first few pages and, politely but firmly, slid it back across the desk to me and said:

“Jim, I don’t care how much work you have done, how much stuff you have put out, how many events you have run, et cetera. I just want to know what value you have added to my business.”

I went back to my office and talked with my colleagues and we realized that we had no clue in relation to his question. The real stinger came when another CEO said: “When you tell me all the things you have done, I see you as a cost centre. It’s only when you can show me results linked to my organizational objectives that I see you as a value-adding centre.”

So I enrolled in a Masters degree by research doing a thesis on evaluation of media and communication (1992-93), and in 1994 was one of the authors of the International PR Association (IPRA) Gold Paper on Evaluation. And it progressed from there. In 1995 I bought the Asia Pacific franchise of CARMA International and got out of the agency business to focus on measuring and evaluating communication campaigns.

TMS: What course of study did you follow? What would you recommend for today’s students?

JM: After doing a BA majoring in journalism and media studies, I did my Master of Arts by research, and then did my PhD in media research (evaluating media impact).

Any and all education is useful. But I would certainly recommend graduate education of some kind for practitioners. Today, the C-suite is full of MBAs and Masters in accounting, engineering, and so on. A basic undergrad education doesn’t cut it anymore. I don’t think a degree in PR is necessary. I never studied PR.

Two things are vital for an education for public relations research and evaluation:

(1) Understanding communication and media in the broad sense (i.e., all types and channels) because that’s core to all PR and corporate communication work, and

(2) Understanding the business, industry, or sector you work in.

So that might mean a communication or media degree and/or study in finance, business, environmental science, or wherever your passion is.

TMS: What’s so special about measurement and evaluation? Why are you doing it instead of something else?

JM: I didn’t have anywhere else to go! No seriously, measurement and evaluation is not an option. And it’s not something you tack on at the end of projects. Unless we are a mind reader, we don’t know what to do in communication until we do some measurement and evaluation (e.g., of current awareness, perceptions, attitudes, preferred channels, etc.). Then we have no idea whether what we did worked unless we do some measurement and evaluation.

If you don’t do M&E, you are either very arrogant, believing you instinctively know how people’s minds work, or you are a prophet. And there are not too many prophets in PR or the measurement supplier market.

TMS: When a client or your boss asks you to do measurement or evaluation in a way that you know to be misguided, how do you handle it?

JM: Practitioners struggle every day with this one. I don’t think a young middle-career PR practitioner with a BA in something can handle this type of situation. And unfortunately there are still no shortage of managers who want AVEs or just raw metrics such as how many times my message was distributed.

My tips are:

(1) Get the qualifications to have ‘cred’. When I was a young whipper-snapper, no senior management took any notice of me. But when I walked into the office as Dr. Macnamara, even the MBAs and lawyers took some notice. You don’t necessarily need a doctorate, but get relevant qualifications;

(2) Learn about research methodology so you can talk with authority. The international work on standards is a big help, but read, learn, and study to expand your knowledge;

(3) Learn and practice diplomacy. We are counsellors, not directors, so we have to bring people with us on a journey, not browbeat them; and

(4) Dye your hair grey if you are getting desperate for some respect.

TMS: Suppose you have to address a tough audience about a tricky project. What A-game presentation techniques will you bring to the meeting?

JM: I’ll engage the audience in the project. Share the background with them, define the problem, and then invite their input. It flatters them, shows them respect, and taps their expertise. And—you never know—they might actually have a better idea than you did. And, if you do use some of their ideas, even a few, they will have ownership and have no option but to support the project.

TMS: What are your favorite measurement tools or projects?

JM: After starting out as a quantitative researcher, I’ve become a big fan of qualitative research. There is no shortage of metrics and analytics (numbers). But, while management loves them (because most of them are accountants, engineers, etc.), numbers are highly reductionist. They compress complex information into a single unit, such as a reputation rating of 7 on a 10 scale. What does that mean? No one walks around saying I’m a seven! Why did they rate you a seven?

What did they like? What did they not like? What would they like you to do next? That’s the real juice that gives you insights and informs your strategy.

So I like a few numbers to pacify the positivists, and lots of qual research to dig underneath the surface.

TMS: Tell us a story of when you used measurement or evaluation to significantly improve a client’s program.

JM: I’ve got the perfect example for you: My recent article in The Measurement Standard on a project in which research substantially changed and improved a major health campaign.

TMS: Where are measurement and evaluation going?

JM: I think they are going in the right direction—but I wish they would hurry up a bit. We have been on this issue for 40 years. I’m dreading that my grandchildren will ask me about AVEs!

In terms of specifics, no one has a crystal ball, but here are a few observations that are based on current indicators and trends in other sectors:

(1) There will be consolidation of the supplier market over the next five years. There are literally hundreds of tools in the market and dozens of suppliers. This trend is already evident and will continue. While this will reduce choice, it will give suppliers scale to invest and improve their products;

(2) Hopefully the push for international standards will not stall and advance on to the hard questions such as how to measure impact, influence, and value. By now we really ought not to be still arguing about reach and impressions;

(3) The borders of the PR industry will increasingly blur and dissolve and other disciplines will enter this space even more than before. As governments and corporations discover the importance of engagement to redress loss of trust and disengagement by citizens and consumers, specialists in consultation, behavioural insights, ‘big data’ analysis, and so on will take the lead.

There is no reason why PR cannot compete in this space, but it will take a significant upskilling and expansion of vision. Counting clips or even views, likes, and follows is not going to do it.

TMS: And finally, if you could invent one measurement or evaluation tool to accomplish anything, what would it be?

JM: First, I don’t believe in fairies. There’s no one tool that will do much at all. It’s like imaging a carpenter, a plumber, or a mechanic with one tool. Imagine saying to a carpenter go build me a house and pick just one tool to do it with! We need a suite or toolbox of tools to suit various jobs.

Second, I don’t believe there is a need to invent any new tools. In fact, good advice for the industry would be to stop inventing measurement and evaluation tools. There is a very sophisticated social research and market research industry out there with a range of reliable, robust quantitative and qualitative research tools. Inventing new methods and tools just adds to the complexity and confusion of the sector and many of them are highly questionable. We can measure most if not all of what we need to measure using surveys, interviews, focus groups, content analysis, ethnography (systematic observation), statistical analysis, and so on. We don’t need another ‘black box’ that automatically churns out PR Value to the power of 10.

TMS: Always a pleasure to talk with you Jim. Thanks for an inspiring interview.
###

Bill Paarlberg
Visit me on

Bill Paarlberg

Bill Paarlberg co-founded The Measurement Standard in 2002 and was its editor until July 2017. He also edits The Measurement Advisor newsletter. He is editor of the award-winning "Measuring the Networked Nonprofit" by Beth Kanter and Katie Paine, and editor of two other books on measurement by Katie Paine, "Measure What Matters" and "Measuring Public Relationships." Visit Bill Paarlberg's page on LinkedIn.
Bill Paarlberg
Visit me on
728 Ad Block

Related posts

728 Ad Block