 |
 |
 |
| Vol.
5, No. 7, December 7, 2006|
To The Editor
| Subscribe | Back
Issues |
MeasuresOfSuccess.com | Masthead |
Advisory Board | Reprint
Information | |
||
To return to the current issue's contents page, click here. To return to the contents page of the issue that this article appeared in, click here. Comments
Please! |
Measurement
In Educational Environments
by Katie Delahaye Paine Crass commercialism has arrived at the door of Academia. I'm sure that's what many academics think of the emphasis on marketing that has been sweeping through campuses across America. But the reality is that more and more educational institutions are realizing that managing their reputations is critical to their survival. You can have the best faculty in the world, but if no one knows it, you're not going to attract students. Academics are learning that in order to manage they need to measure. So PR measurement is making it onto to-do lists of communications and public affairs officers in colleges and universities of all shapes and sizes. There are several interesting elements that make research and evaluation unique in an academic environment. Rule
#1: For starters, you're dealing with an academic environment, where people do research for a living. These people are smart, and they didn't get PhDs by being shy about it. They will challenge your statistics and your calculations in ways that very few people in the corporate world ever will. They live and die by the accuracy of their data, so yours better be just as accurate and defensible as that of any of the Master's students they advise. So watch your R factors when you're doing those correlations. Rule
#2: Unlike corporate America, where most of my clients run screaming from the room the moment the data gets too deep, people in the university environment make sure they always have lots of it. If you need to know the average SAT score of every freshman class for the past five years, they've got it. If you need to know how many alumnae come back for every football game, chances are someone has it. And it's not just the professors. I've spoken to a number of university presidents and they can rattle off statistics like it's not to be believed. And the good news is that if you want to see what impact your PR program is having on attendance at football games, or on the ratio of application downloads to SAT scores, the data is there. Rule
#3: The head of almost any educational institution can tell you off the top of his or her head basic measures of success, like their ranking in the US News and World Report survey, their alumnae donor rate, their retention rate, average SAT scores and their discount rate (the difference between full tuition and what students are really paying). These are the stats that university presidents live and die by. Rule
#4: Ultimately the goal of any university communications program is to contribute to the education of the students. That is the bottom line. That's why you need to raise the money, get the publicity, boost the rankings, or have that press conference. Ultimately if you're not educating students, the institution will cease to exist. Rule
#5: Another element that makes measuring in an educational environment a bit more challenging is the number of specific audiences you need to address: Faculty, staff, students, parents, alumnae, town officials, state and national politicians. The list isn't necessarily longer than for your average multinational corporation, but the needs and interests are very different. In a recent presentation at the Whittemore School of Business and Economics at the University of New Hampshire, Steve Reno, Chancellor of the University System of New Hampshire, listed ten different stakeholders with some six dozen different agendas that he needs to juggle on a daily basis. Rule
#6: Few educational institutions budget anywhere near enough for communications and/or marketing/public affairs, never mind putting measurement in their budgets. But the good news is that because so many of the metrics are readily available it doesn't necessarily have to cost a lot of money to put a measurement system in place. Seven Steps to Academic Measurement Excellence The following seven steps will guide you through the process. Step
1: The first step is to understand the demographics and psychographics of your various audiences. Remember that your multiple constituencies are not homogenous. They each have different needs, wants, expectations and desires. There are new students, returning students and former students. You have tenured faculty, part-time faculty, instructors, etc., all with different needs, wants, and expectations. So the first step is to figure out just who it is you need to talk to and have relationships with. Typical audiences within the academic environment include:
Next you have to prioritize which relationships are most important. To do that, you need to understand how a good relationship with that constituency benefits your institution, or how a bad relationship can hurt it. Good relationships with students might keep peace on campus, but in reality, given their predilection for blogging, the ultimate outcome of a good relationship with students might be to to have student blogs reflect that good relationship to the world. So make a list of all your constituencies and next to each one list the benefits that a great relationship with that group would bring to your institution. Then gather the senior leadership team and key communications people within the institution and get them to prioritize the audiences. The way we do it, everyone gets 10 colored dots and each one is worth $100,000 of communications budget. (Just think of it as Monopoly money.) I then ask all participants to "spend" their budgets in the ways that they think are most appropriate. The audience with the most dots is clearly the most important. Step
2: The most important thing to remember about any measurement program is that you become what you measure. Those metrics that you define as important will be the ones that everyone in communications will attempt to achieve, so getting them right is crucial. One exercise we put our clients through is to get them to close their eyes and imagine their President delivering a case of Don Perignon to their desk in gratitude for the best year ever. Then I ask them to tell me what has changed, or what is different from a year ago. Start by going back to your list of the benefits a good relationship brings to the institution and see if any of them would make good objectives. For example, if your institution's number one priority is to grow the student body while maintaining academic excellence, then a typical metric might be media visibility vs. increase in the average SAT score of applicants. If your mission is to build the endowment, then the metric might be the percentage of alumni who donate, or the increase in average alumni donor amount. If you rely on the Web to attract applicants, you might want to make Web traffic and application downloads a key metric. Step
3: When you're thinking about metrics, remember that audiences do not live in a vacuum, nor do they only hear your messages. In today's society any one of your students/faculty/alumnae, etc. is hit with some 5000 messages a day from a wide variety of sources. Some, like advertising and direct mail, you can control. Others, like the media, you can't. A good measurement program looks at as many different influences as the budget allows. At the very least you should take into account what your students/faculty/alumnae, etc. are seeing in the media, what they're exposed to online, what they're hearing from you, what they're taking away in terms of impressions, awareness or understanding and ultimately what actions do they take, e.g., download applications or brochures, donate, etc. All these various influences affect your relationship. So it's necessary to understand and quantify those influences if you want to understand why the relationship is improving or declining. Step 4: Establish a benchmark If I tell you that 35% of alumnae donate to your university, what does that tell you? Essentially it says that 65% aren't donating, but it doesn't tell you whether that is a good number or a bad number. Do your peer institutions have higher or lower rates of alumni giving? Is your rate of alumni giving going up or down? Since measurement is a comparative tool, you need to figure out what you are benchmarking your results against. It could be a peer institution or a competitive institution. In the non-profit world, unlike corporate America, cooperation between institutions is much more common, and institutions may be able to partner on the research in order to have something to compare their results to. Whether you select competing institutions or peer institutions, try to limit the number of entities in any given study to no more than five. Three is ideal, anything more than five becomes unwieldy. Step 5: Pick a measurement tool Once you have your objectives clear and your benchmarks established you can decide whether you're going to be measuring outputs (e.g., articles that appear in the media); outtakes (what people are thinking), or outcomes (what people are doing). Depending on what you're measuring you will either need to analyze your media coverage or survey your students/faculty/alumnae, etc. or preferably both. Measuring Outcomes You don't have to analyze the world, just those publications or media sources that are most important to your audiences. So if faculty is your number one priority audience, you need to make sure you're measuring academic journals, as well as mainstream media. If students are the number one audience, you need to remember that this group watches very little mainstream media and gets most of its information from online sources, so you should be monitoring forums like MySpace. Once you've established a key media list, make sure you have access to all articles in those media outlets, either via your own subscriptions or via clipping service like Bacons, Burrell's, Factiva, Lexis, CyberAlert or CustomScoop. One thing to remember is that communications professionals do not read the media like normal human beings. We are far too quick to spot a key message (particularly one's own) and are much more sensitive to reporter's opinions than are normal human beings. Ideally you should find a member of your target audience to analyze the media. Or, if your volume of coverage justifies it, you can use one of half a dozen automated content analysis programs such as Cymfony or Biz 360. To accurately gauge the impact of your media efforts, you should analyze all mentions of your institution as well as your peer institutions for the following criteria: 1. The main subject 2. The type of article it is: Opinion, feature story, Q& A, letter to the editor, etc. 3. The visibility of the institution within the article: Was your institution the focal point, or did it just receive a minor mention? 4. Who, if anyone, was quoted in the article? 5. The tonality: Did it leave the reader more or less likely to join your institution? 6. The type of media in which the article appeared: TV, magazine, business press, etc. 7. What, if any, messages were communicated? 8. How was the institution (and its peers) positioned on key issues such as "good value for the money," "effective advocate for the industry," or "responsiveness." If you don't hire an outside firm to conduct this research, there are several software programs out there that make it a lot easier, including CommAudit, PRTrak and KD Paine & Partners' own DIY Dashboard. Measuring Outtakes To measure outtakes you need to do a survey to understand how your students/faculty/alumnae, etc. feel about their relationship with the institution. Drs. James and Laurie Grunig have developed a standard survey instrument that can be used with a high degree of accuracy to judge the health of your relationships. The methodology is based on the notion that a relationship has five essential components:
For each of these components, Linda Hon and James Grunig have developed specific agree/disagree questions. Survey respondents are asked to use a 1-to-9 or 1-to-7 scale to indicate the extent to which they agree or disagree that each item listed describes their relationship with that particular institution. You can find these questions and more details about this research in Grunig and Hon's paper "Guidelines for Measuring Relationships in Public Relations" at the IPR website. For a complete list of survey research options, read Dr. Don Stacks' book Primer of Public Relations Research, available at measuresofsuccess.com. Measuring Outcomes Audience member behavior can take many forms – renewals, Web traffic, email responses, phone calls, attendance at trade shows, votes. All are relatively easy to measure, once you have a tracking system in place. The key is to keep track of them on a monthly basis, and then compare the results to your media activity and, ideally, to the attitude research. With sufficient data you will be able to see correlations between activities in the media, communications with your students/faculty/alumnae, etc. and behavioral outcomes. Step 6: Analyze the data and glean insight All the data in the world is simply trivia if you can't draw some conclusions from it. So when the data is in, look for trends, differences between new students/faculty/alumnae, etc. and old students/faculty/alumnae, etc., males and females, and by all means look most carefully at the bad news and the failures, because that is where you will learn the most. Step 7: Make changes and measure again Once you have your relationship measurement program in place, you need to make it a regular, on-going part of your communications process. When the data comes in, you learn from your mistakes, make changes, and then see the results in the next reporting process. One caveat: Make sure the data is available when you need it. If you do all your budgeting and planning in August, having end-of-year data does you no good whatsoever. The data should be fresh just as you are beginning your plan. How to budget for research in an academic environment The biggest question is always how much should you budget for measurement. There are two ways to figure this out:
One advantage in an academic environment is that you have student interns that can do a lot of the coding if need be. The reality is that having data on which to base decisions is worth every penny, and almost always the cost of not measuring is higher than the cost of measurement. For an example, see the companion case study to this article, "The High Cost of a Higher Education in PR."  |
|
||
|
|
Three Reasons Why You Should Subscribe to The Measurement Standard: 1. You’ll learn how to use hard numbers to prove the results of your PR efforts. (Plus, it's free.) 2. You’ll learn which are the right vendors for your measurement projects. (Yes, it's free.) 3. You’ll learn how to design your program right from the start to be easily measureable. (Plus, yes, it's free.) |
|||
|
|
|
|||
|
|
||||
|
|
||||
Struggling
to set up your measurement system?|
51b Durham Point Road, Durham, NH 03824 |
|||
