I always planned to use real estate space on our collective PR Conversations platform to spotlight some exemplary Canadian PR practitioners. When it comes to an overview of the Canadian measurement scene, few can match the depth and breadth of knowledge of Alan Chumley. Instead of an interview, I offered Alan a guest post, where he could write (in his own words) about measurement in Canada. So, gentle readers, I present to you Alan Chumley, who very much welcomes comments and feedback from our international audience of PR academics and practitioners, colleagues and friends. JG
* * *
I was thankful and honoured when PR Conversations invited this Measurement PRoponent / PRomulgator to opine on the current state of the art (or science?) of measurement in Canada. While it’s a topic that could easily spawn a healthy hundred-plus-page graduate thesis, the following is a reasonably succinct, high-level, non-exhaustive horizon scan of measurement in Canada, broken up into bite-sized morsels to cover the following topics: industry associations; Canadian vendors; U.S vendor spillover; the client context and the spectrum of “outs”; standards, patterns ‘n packages; the agency sphere; homegrown, thought leadership on measurement writ large; Canadian measurement events and conferences; education; awards; the wrap; and the ask.
Industry associations
Many Canadian practitioners look south of the border to the Institute for Public Relations’ Measurement Commission for example, for institutionalized thought leadership on measurement and across the pond to the less prolific, U.K.-based-but-global Association for Measurement and Evaluation of Communication (AMEC). (I’m a booster and disciple of both.)
Canadian associations, perhaps most notably the Canadian Public Relations Society, plus the Canadian Council of Public Relations Firms and the Toronto (and other Canadian) chapters of the International Association of Business Communicators have played a role in advancing the measurement discourse in Canada. They’ve done so through hosting events and through endorsing methodologies.
Case in point, the Canadian Public Relations Society’s measurement committee rolled out its Media Relations Ratings Points (MRP) in April 2006. Endorsed by both IABC/Toronto and the newly minted Canadian Council of Public Relations Firms* it was launched with great fanfare at a very well-attended event, and the method has since seen a fairly healthy take rate. The CPRS Measurement Commission* apparently worked for three years on MRP. They, along with NewsCanada, are to be congratulated for recognizing and working toward solving a distinct need in the Canadian marketplace with, at least, a simple, cheap editorial scoring system for, largely, MarComm-driven coverage. It also got Canadian practitioners en masse talking about measurement again; at least now we’re all drawing from data sources consistently. However, even by the admission of the measurement committee, it’s not without its limitations. It works reasonably well in a very specific context for which it seems to have been designed, beyond which one would still need to look at undertaking additional research if one wanted to go deeper than an 80 per cent score and a simple CPM calculation…and we all know we should be doing that.
Canadian vendors
Picking up where MRP leaves off, Cormex Research has a healthy grip on the Canadian media content measurement and analysis market. The other key Canadian player, Ottawa-based MediaMiser, uses a hybrid human and automated system for near real-time analysis and reporting. They offer slightly different services, tailored for different client needs. I’ve seen colleagues and clients use MRP in concert with deeper content-analysis and data-cross tabulation.
U.S vendor spillover
MB Precis** completes the odd project—ranging from content analysis to full-impact studies—in Canada, but based out of its offices in the U.S. Similarly, Carma, operating under Montreal-based Leger Marketing in Canada is an occasional “also ran.” The organization formerly known as Delahaye-turned-Observer Group-turned Cision owns a media monitoring arm in Canada, but little of the type of measurement that Delahaye does in the U.S. has crept north of the border.
While I’m encouraged by the amount of in-depth media content analysis that’s being undertaken, at the end of the day it remains a measure of output until—or unless–it’s linked with other sources of data such as polling data. I don’t see much of that happening. Also, I don’t imagine there’s much in the way of Market Mix Modeling or Six Sigma (specific to a communications context) going on in Canada.
The client context and the spectrum of “outs”
In 2004, Watson and Simmons published a study that, among other notables, pointed out that 89 per cent of practitioners are measuring output, but only 32 per cent are measuring bottom-line business or organizational impact. (I suspect this later number to be even lower in Canada). There are several other major studies out there before and since then have come up with similar proportions; however, I suppose the key point here is that, with well-earned respect to notable exceptions, and while there is some excellent work going on in Canada, we continue to be hampered by an output measurement mindset and are challenged with moving people to outtake, outcome and outgrowth. (Perhaps paradoxically, but encouragingly, Canadians have largely rejected the use of “advertising equivalency” numbers). The output-outgrowth war is a battle we measurement mercenaries fight daily. We know there’s more to communications than media relations. It’s about relationships, reputation and everything that’s done to drive and manage both. All of which can and should be measured. I can’t tell you how many times I’ve had a conversation where I’ve asked a client or a prospect, “Did you know that we can measure x?” The result is overhead light bulbs and raised brows at pitch time, but knee-jerk reaction come execution and budget time. An issue not exclusive to Canada, I suspect.
Standards, patterns ‘n packages
I’ve noticed, particularly but not exclusively on the client side, that there’s a tendency out there to want to (over)simplify measurement solutions into patterns and packages such that a client of a certain size with a certain need could be slotted into a bronze, silver, gold or platinum measurement package. The easy answer isn’t always the right one in measurement any more than it is in building one communications plan to suit all client needs (not to mention their reporting structure, operating environment, management style, communications department culture, PR practitioners’ roles, their research orientations and the prevailing model of PR resident in the organization). I suspect I’m preaching to the converted, but, I evangelize to account teams internally and clients and prospects externally that each measurement program must be built to meet specific needs. Adhere to an industry standard? Heck no. There isn’t one and nor should there ever be. Regard industry best practices (such as those that come out of the Institute for Public Relations’ Measurement Commission)? Absolutely. It’s a tough sell, but something we need to continue to promote.
The agency sphere
Canadian agencies just don’t have nearly the research and measurement resources that our brethren in the U.S. do. For example, in Canada, with the exception of Doris Juergens, partner and senior vice president of National PR’s Research, Information Services and Training (who also sits on the CPRS measurement committee) and me at Hill & Knowlton Canada, you’d be hard pressed to find another senior-level position that is devoted exclusively to measurement and/or research and measurement; roles that we see more commonly in the U.S. It’s nothing to do with how measurement is regarded in this country. It’s more to do with what every Canadian organization that also has a U.S. counterpart suffers from. The old order of magnitude issue where one can typically expect to knock at least a zero off the client’s budget.
Homegrown thought leadership on measurement writ large
In my view the list of Canadian measurement thought leaders is limited to Cormex’s Andrew Laing, National’s Doris Juergens, Tudor Williams, and IPR measurement commission member Fraser Likely; however, none have yet emerged nearly as visible and vocal as is KD Paine in the U.S. and globally.
Canadian measurement events and conferences
You’ll often see some of these thought leaders presenting at events with a measurement focus. Though they certainly don’t compare to the Institute for PR’s annual Measurement Summit, there is the odd conference dedicated specifically to measurement, including the 10th Annual Communications Performance Measurement conference. Generally, though, measurement is relegated to a one-hour session at the end of a conference with a broader context such as media relations or government communications. It’s wonderful that measurement is included in these events…but they are usually scheduled at the end of the conference when many attendees are zipping up their bags and dashing to catch a flight. A shame.
Education
Education has an important role to play in how measurement is regarded in this (and in any, I suspect) country. Unlike many other markets, undergraduate PR education has, with very few exceptions, been almost exclusively relegated to strictly vocational training at Canadian community colleges. While there’s a market and a place for that type of program, I fear it tends to marginalize how research and measurement is taught, if it is at all. Based on my admittedly anecdotal research of most of the major undergrad-level PR programs in this country, very few have a stand-alone course dedicated to research and or measurement. If courses specific to fundraising, photography and event management can make the curricular cut, then surely there’s a place for research and measurement.
Awards
I recently had the pleasure of judging IABC/Toronto’s OVATION Awards submissions. For the most part the quality and creativity of the projects submitted was top calibre. What was disappointing, though, was how the majority of the projects measured success. For example, if an objective involved raising awareness, then there better be a measure to gage it. Generally, there wasn’t. In my view there’s a need to call for:
1) More rigorous measurement criteria against which to judge submissions.
2) Canadian awards specific to measurement, similar to the IPR’s Jack Felton Golden Ruler award, and AMEC’s awards, a topic that I’ve written about previously.
3) More Canadian submissions to the Golden Ruler. The behaviour of raising our measurement game needs to have more incentives and be adequately rewarded.
The wrap
It’s only through thought leadership, conferences, awards and increased academic curriculum attention, plus augmented involvement from industry associations and a collective dialogue among stakeholders—including academics, students, vendors, associations, and both agency and client-side practitioners—that we will continue to productively push the measurement agenda forward in Canada. It’s progressed much in the last 15 years, but there’s a long and winding road ahead.
The ask
While these may be issues that are sources of similar concern in other regions of the world, I wonder: To what extend do PR Conversation readers feel that they are they shared and/or more or less acute in different countries?
Alan Chumley
Director of Measurement, Hill & Knowlton Canada
*Disclaimer: my employer, Hill & Knowlton Canada, is not currently a member of the Canadian Council of Public Relations Firms or the CPRS measurement commission.
**MB Precis and my employer Hill & Knowlton are both owned by the same parent: WPP Group.
For more Alan Chumley:
1. My Proponent / PRomulgator blog
2. H&K blog
3. AMEC’s blog
Blog posts on Alan Chumley opines on “the current state of the art (science?) of measurement in Canada:
The state of PR Measurement in Canada
Interesting analysis of Canadian research
Six Sigma Green Belt Certificate Program at the University of St. Thomas.
I agree that “I don’t imagine there’s much in the way of Market Mix Modeling or Six Sigma (specific to a communications context) going on in Canada”. It depends on what is being measured and how it is being measured. Six Sigma Green Belt Certificate Programa> at the University of St. Thomas located at their Minneapolis campus.
Quasi-live blogging from Canada’s Measurement Event?
http://alanchumley.wordpress.com/2007/09/07/quasi-live-blogging-from-torontos-measurement-event/
Thank you for those dates, João. Now more choice! Benita is right. I trust that this London conference can fill the “big shoes” that Toni wore with the Trieste conference in 2005 – and that if we are going to institutionalize anything, it’s the partying ability of the South Africans!
Toni, I agree with you on the questions you have concerning the background/role of the Head of PR/C. Should the head be staffed by the CEO/Chairman’s personal choice (someone brought in from the outside; some one who has reported to the CEO in previous organizations; someone with existing chemistry)? Should the head be staffed by a “career” PR expert (someone who has worked their way up the ranks, in this organization; in others)? Should PR be seen as short-term stepping stone (for someone on the executive fast track)? Should there be a strategic head (personal advisor to the CEO) and an operational head (runs the shop)?
I’ve seen all, and various hybrids. Sometimes a particular model will work; sometimes the same model won’t. What we should keep in mind is that the same models apply to other corporate functions. Besides research on our own function, we should be looking at the others as well.
I didn’t mention it, but the Festival and Conference for Global Alliance for Public Relations and Communication Management will take place in London, England next June.
May I suggest that you to consider also the 5th Edition of the World Public Relations Festival between 23-24 June 2008.
Next year this gathering promoted by the Global Alliance for Public Relations and Communication Management member associations will be organized by the Chartered Institute of Public Relations and will be discussing “The Public Benefit of Public Relations”.
I’m sure it is not that Fraser can’t make up his mind. I think it is that he doesn’t want to make up his mind–he wants to attend them both! And so do I–but for us intercontinentals from the Third World the exchange rate is a killer.
I remember 2002 when Bled and Euprera were together. What a wonderful conference it was. And how much you made us from faraway places feel at home–although some of us came with strange ideas. And 2005 when the 2nd World PR Festival was back-to-back with Bled. We came in droves. And taught the Europeans how to party, now didn’t we Fraser?
It is obviously less than ideal for the Europeans to have the conferences close together. But what about say once in 3 years to assist us ‘financially disadvantaged’ ones. (It doesn’t need to be Bled and Euprera each time. There are also other conferences to back up to. For instance, the Corporate Communication Institute’s conference takes place just one month before Bled in the UK. So close and yet so far). But then, we all have a wishlist, don’t we……
Euprera, as you know, has always been involved in the Bled Symposium and I know they were well aware of the Milano Congress theme, which they selected about eight months ago, when the Bled theme was decided…. Maybe it is good that similar themes are discussed at three months distance…also, the institutionalization has many other facets beyond involvement in corporate processes and relationships with other management functions (for example, how institutionalized is a function when the director of communications leaves his corporate post as the CEO changes job?..or how institutionalized is a function when all the organization really wants from its pr director is facilitated access to specific political or media circles?…or how institutionalized is a function when personal influence and good networking is most of what a director of communication has to offer? Is the institutionalization process going to freeze the development of the profession through red tape? Is the institutionalization process the beginning of the end of a profession and the pure consolidation of a management role?
As for Bled, personally I understand the theme to be more like: let us revise the stereotype that public relations belongs to marketing and at the same time carefully monitor marketing’s increasing advances towards public relations…I sense that the reference to the other managerial functions was a post thought to discourage what could easily become a ping pong game between the two disciplines… But I agree, similar things happen often amongst associations who do not speak to each other. In this case however Bled is a private organization and not an associations, has a great reputation, and will certainly not change its mind simply because there might be some overlap… Besides….the more the issue is discussed the better and Milano will have the benefit of leveraging the contents discussed in Bled…
Toni, so much to chose from …
Bled, Slovenia early July 2008 with the theme: “Public Relations and Marketing: A Relationship for Re-examination? Public Relations Among the Functions of Management.” I assume the “Public Relations Among the Functions of Management” will draw considerable interest.
Then there is Euprera in Milan, Italy in October 2008 with the theme: Institutionalizing Public Relations and Corporate Communication.” I assume “institutionalization” includes involvement in corporate processes such as strategic management as well as relationships among other management functions.
Very similar themes! Same neighbourhood of northern Italy! A conference goer/tourist could beneift from two great events as well as take in the magic of northern Italy in a few weeks time.
Except, they are 3 months apart!
Too much choice …
Fraser
my friends,
I have decided to dare…and… barge into this fascinating canadian caucus and file a comment, also in the hope that others from other countries and experiences will want to weigh in and contribute to what is probably the most amazing and interesting dicussion on the topic I have ever witnessed.
In following the debate, I was frequently stimulated, but Fraser’s last comment and his reference to the coming bled and milano (euprera) conferences really urge me to put in my two-cents.
My professional upbringing is principally based on seven solid years (1963-1970) of executing 3M public relations in Italy. Very much of this (not all..) in the area of product publicity. Even in those years my excellent St.Paul senior colleagues (John Verstaete and Paul Brown, subsequently Bob Peterson..) were much more interested in my evaluation of the impacts on customer behaviours (admittedly business to business..) of the outputs and out takes of our work, rather than the measurement of column inches.
I was then fortunate enough, having moved in the meantime to consultancy, to have worked for more than 10 years each (20 years in one case!) for three mnc’s (Philip Morris, American Express and AT&T) whose international public relations directors (the latter was Frank Ovaitt, today my coblogger here and effective ceo of the ipr..) were almost totally disinterested in output and out take measurement and concentrated their attention on evaluating outcomes, thus allowing significant time and resources for the exercise.
In the mid eighties, as Ceo of the largest italian pr consultancy, I launched an internal program in which any activity which implied resources of more than 50.000 euro, needed to adopt a pre-post qualitative outcome analysis methodology even if the client was not interested.
More recently, say in the last 10 years, just about every project I am directly involved in requires not only outcome but also outgrowth analysis (by which I mean that that single program is measured in its impact on key stakeholders, but also in the quality of the organization’s relationship systems as well as their reputation).
Not surprisingly, I have recently sold Methodos, the change management consultancy of wich I was chairman and minority shareholder to a….research company: Doxa, Italy’s most reputed and oldest market and social research organization, and for the last seven months I have been having a whale of a time in inventing, testing, discussing new ideas in order to devise new products and services which, despite all the potential and real conflicts of interest of which I am well aware, may effectively integrate consulting and research cultures for our clients benefit in measuring (always) and evaluating (where possible) the four levels of output, out take, outcome and outgrowth.
In October of 2008 Milano (IULM University and Ferpi) will host the annual Euprera congress
on the theme ‘the institutionalization of public relations and coporate communication’. Euprera is the european association of public relations research and education: an excellent organization which has done wonders to create a community in which schoilars, teachers and professionals work from many countries work together to forward our body of knowledge.
There are many aspects related to the institutionalization issue (a very ambiguous concept this, which raises many questions, both good and bad..)but, in my view, the central one has to do with the (if you prefer..) the consolidation of the public relations function inside any form of organization. And there is no doubt in my mind that research (including evaluation and measurement, but not only: think of the listening to stakeholders role…)is firmly embedded as a cumsubstantial (does such a term exist??) part of our day to day activity. No practice is acceptable if it is not related to a predefined objective. If we start from there, the rest comes along.
This is why I thank Fraser for having mentioned the Milano congress. If canadian, american, australian, new zealand, south african, latin american professionals, scholars and educators decide to join their thoughts with those of their european colleagues we could be hopefully leading to a truly relevant event.
Craig, I’m very happy to hear that you will devoting some research time and energy to the PR/PA/Communication field. With tongue-in-cheek, I’ll say that you spent too much time with the CI field, those dark arts!
It’s pitiful the number of Canadian academics doing serious PR/PA/Communication research. Even in related fields that affect our field, where are the Stanburys, the Gollners, the Robinsons, the Taylors, or even political science scholars like Paul Pross?
What I have really appreciated about your research, Craig, was that you worked with practitioners, giving your work a practical and applicable side. Carrying on that tradition, I have, together with Stacey Smith of Jackson, Jackson and Wagner agreed to sponsor the Jackson-Sharpe Award at the annual International Public Relations Research Conference organized by Dr. Don Stacks in Miami each March. The award will go to the best academic-practitioner co-produced research.
(As an aside, thinking of Pross, a few years ago I gave a speech to a CPRS audience in Halifax based on my own grad work on societal issues management. Both the late Ron Pearson then in the PR Program at Mount St. Vincent University and Paul Pross from the Political Science Department at Dalhousie were in the audience. Pearson of course was becoming the thought leader on PR and ethics and Pross was an international authority on interests groups and political systems. A most interesting lunch conversation followed! My point: there are rare opportunities for practitioner / consultant / academic interaction – and we need more.)
So happens Craig, I have your 1995 Public Relations Review article open on my desk as my current research is looking at the effect of a corporate Balanced Scorecard on PR department “comprehensive” scorecards. Thinking about what you last posted – as additions to Alan’s list (1. What gets measured gets done. 2. What gets counted, must count.), I add one more mantra:
“What gets measured must have a consumer/champion for each metric.”
By consummer/champion, I mean an executive that needs to consume a specific measure to further his or her group’s work and therefore become the champion or co-champion of that measurement. That said, for example, a CEO may rather consume PR function productivity, overtime or retention measures than media content analysis measures. Or relationship measures rather than web site usage measures.
My point is that we are concerned with output (particularly media) and occasionally outcome measures, where the C-suite may well be concerned with other and higher-order function performance measures. Most of our present day measurement in PR examines the role of practitioner as technician not as manager.
The difference is spelled out in two books: The HR Scorecard by Dave Ulrich et al; and How to Measure Human Resources Management by Jac Fitz-enz. The first examines “strategic” measures, the second “tactical” measures. Change HR to PR and it all applies.
Next year’s Bled and Europra have themes that may be of interest to your research efforts, or to Alan’s or Terry’s. Would be nice to have Canadians represented there.
Best.
Fraser
A 12-step program works for AA, so why not measurement? I quite like your 12 guidelines. I look forward to seeing where your sabbatial research takes you.
Hi Alan: I like your mantra. All of these echo closely to the things my research has uncovered. I’d add two more items to your already strong list (maybe we could add a few others as well — these are just off the top of my head):
1. What gets measured gets done.
2. What gets counted, must count.
Many years ago (it was in a book I published in 1995 to be exact — I’m sorry to be showing my dated-ness here), I had published in a chapter a list of the guidelines my research suggested organizations must take to successfully utilize measurement effectively within their PR/comms functions. That list included the following:
1. Handle measurement as a change (note my earlier entry on measurement for improvement) strategy
2. Create a vision for performance improvement through PR measurement
3. Get senior management buy-in and involvement
4. Aim initial measurement efforts at high success probability areas
5. Strategically manage PR measurement planning and implementation process
6. Choose your measures wisely
7. Be alert to political ramifications of measurement
8. Grow the measurement effort from the ground up
9. Build ongoing communications around measurement and its linkages to PR strategy/tactics/goals
10. Provide needed training and support (most people don’t know how/what to measure — and some people — hopefully a very small percentage in this day and age — still don’t teach it to our college or university PR students)
11. Evaluate your measurement system (ironic, eh?)
12. Prepare to accept delayed gratification (the beneficial results of effective measurement may take time to experience and observe — just like the outcomes of our PR programs which don’t always have immediate impact).
Would these still hold true today? Any thoughts from any of our readers?
I’m hoping to engage a couple of research projects in the next 12-18 months while I am on sabbatical to re-visit some of my earlier research to see if much has changed in the last decade or two. Fortunately, I even managed to keep most of my raw data from surveys I conducted in the US and Canada (primarily — although I have some from other parts of the world as well — just not as much of it), which should allow me to do a bit of longitudinal comparison. In any case, I wonder how much has changed and how current practice will compare to past ones?
On a separate (yet not totally unrelated note), I was so pleased to see groups of PR students recently in the UK, Australia and here at home in Canada (these ones were bright undergrads and grad students at MSVU where I was reviewing their programs), and how mature, professional and eager they were to be in the field. In candid and unscripted conversations, they seemed comfortable with the language of evaluation, measurement and research — although many of these still suggested to me that they awaited opportunities to do it “for real” in their co-op or ongoing organizational roles. If these were indicative of students entering the profession more broadly, then the future is bright for PR.
Although I have been variously labeled as a “grandfather,” member of the old guard, pioneer, foundation builder or other such things in recent reviews of my work in the public affairs field by several prominent writers — all suggesting I am probably out of touch, I have tried to remain actively involved and do generally feel good about developments and the further professionalization demonstrated by PR practitioners around the globe over the last few decades. Yes, there is still work to be done, but every now and then we need to stand back and appreciate how far we’ve come.
Is this your sense as well?
Here, here Craig!
Having done a master’s thesis on the topic (It’s dated and stale, mind you, so I won’t share it), I echo your remarks about progress. While much has been made, it strikes me that not as much has been as was perhaps collectively hoped. I undertook a fairly extensive literature review of both academic and trade sources (and not limited to strictly public relations sources) and found that measurement has been characterized as problematic since at least the 70s. It goes back much, much earlier than that, I’m sure. In fact, KD Paine tells a tale of her father, then the managing editor of Fortune Magazine, who challenged members (of an organization that would later become the PRAS) at an event to “solve the measurement dilema.” That was 1946! I simply got tired of reading the same basic problem over and over.
Again, we’ve come a long, long way and there’s much to celebrate but, like you Craig, I fear sometimes we’re only talking to our own kind more than we’d like and think. My faith in the knowledge level of and committment to measurement out there among the practitioner set rises and falls like some corporate sine wave. However, the distance between apex and valley seem to be declining, encouragingly.
You touched on an item with you scale analogy that we fight daily here in Canada. And that’s the idea that, in my view, measurement is best looked on as a long-term, on-going committment.
Guidelines that I live my measurement life by…my measurement mantra, you might say:
1. The industry must move beyond measuring simple outputs
2. There is no, no should there be a singular industry standard for measurement
3. Approaches to measurement must be evolutionary and flexible
4. Measurement is research, research is measurement
5. Comunications objectives must be linked to organizational objectives and must be measurable
6. Measurement is a long-term committment
7. Measurement must be budgeted for
Thanks Judy. I agree that Fraser is discerning, and has made some very notable contributions of his own through the years to this area. I’ve always thought of the “state of the field” in economics terms, such that we had a supply (and certainly not a demand) problem in PR measurement. There are many people who ostensibly claim to be measuring PR-related phenomena, and many actually are (although not always the best things to measure. In my research, I found that more often they (practitioners, consultants, etc.) measure the so-called “easy” things or things that have been measured in the past. Measurement is rarely “easy,” and requires support through research and evaluation processes, which are also not necessarily easy things to do. Done correctly, these processes enhance one’s legitimacy with clients, and enable our organizations to better achieve their aims (which we hope were also informed through research, evaluation and measurement).
Thinking about PR measurement, I am always reminded of an analogy. Many of us weigh ourselves on a regular basis. I do it because I like to be able to get out of my chair, walk around with less tiredness, or ride faster/farther than before on my bicycle. I measure this variable daily. But the goal of my measurement is not to really understand how much I weighed at a particular time, but rather to understand whether my weight (as measured) was either helping or hindering my primary aims. When I am lighter, everything goes better for me. When it shows me trending/getting heavier, then I need to take corrective actions. Measurement is about the language of improvement. If nothing needs improve, I’d dare say you probably don’t need to be concerned about measurement. If there is room for better PR performance (however it is defined), then more effective measurement can be a difference maker!
I suspect this is all old news to the readers in this discussion, but I am still surprised to find many PR practitioners who know they need to measure but a) have little idea how to go about it, b) will measure for measurement’s sake, c) will measure badly, d) will measure phenomena not actually associated with their project/task or organizational improvement objectives, or e) will uncritically accept others’ measures of their PR activities without having really given substantial thought to it (and this list could go on further as we all know).
It is nice to be able to visit among family. Thanks for allowing me to drop by.
Craig
Dr. Craig Fleisher, welcome to PR Conversations. I’d say you are definitely a part of the family (not just a friend), as both government and academic institutions (in particular, accounting and business departments) are two of the “publics” I work with on a regular basis. Besides which, anyone who has Fraser’s stamp of approval is definitely a thought leader and influencer, as he is mighty picky (ummm…I mean discerning).
I am pleased to be recognized as a “thought leader” in this area, although I guess I am nearly always an outsider looking in since I work in a business school and not in a PR/comms setting. Some of you may also suspect that I am actually coming from a public affairs (usually viewed as government relations) background and have little overlap with PR. Boundary spanning is always a difficult practice to pursue. I just hope if I can’t be part of the family, at least I can still be seen as a “friend” in this discussion.
Having said all of that, I tend to agree with most of the original post, and might actually suggest that things (i.e., practice, art, science, influence, — I could go on ,etc.) are probably worse in reality than many of us think. Being thought leaders one at all — hey, we care about things like measurement, evaluation and research in PR, we are in a small group of practicing, researching, speaking, and/or teaching pioneers who must frequently “shout in the wilderness” to our colleagues out there about the importance of these activities. The earliest pioneers of new lands were also are those usually who were among the first to be found dead with arrows (or worse these days) in their backs…
I’m often accused, probably with some degree of truth attached, of seeing the world of PR measurement, evaluation and research through rose-coloured glasses.
Like many of the others in this discussion, I have witnessed through my research how it can make a significant difference. I have been stating this publicly for decades now, and my research has produced plenty of examples of positive change created at many levels and towards different objectives through effective applications of these activities — I have written several books and numerous articles highlighting some of these examples; nevertheless, I am not too naive to think that this is the norm. Indeed, it is not and I’d offer that there is very little systematic empirical evidence that can be produced to suggest the contrary.
Although I’m still optimistic about the future of measurement (and its close cousins I have also referred to), we didn’t make the degree of progress I had hoped for here in Canada over the last two decades. As long as we still care about it, and have people like those in this discussion keeping it front and center, the future may still be brighter than the past. I still intend to contribute through my research and teaching, and will enjoy learning from my new friends in this discussion and elsewhere.
Dr. Craig Fleisher, Windsor Research Leadership Chair, Odette School of Business, U. of Windsor
Further to yesterday… where I do feel pre-event research is very valuable is in employee communications.
Knowing what employees think, albeit in broad strokes and allowing for a wide variety of opinions, is very useful in planning PR programs, whether aimed directly at employees or aimed elsewhere.
Employees are the most important PR audience for almost any element of any program.
What saddens me (assuming I car enough to be saddened) is the dumb employee research so often done, often but not always by personnel departments.
What I find most useful about employee attitude research is the ability to use them to hit senior management over the head.
BAK
Fraser,
Thanks so much for adding to the list of Canadian measurement thought leaders. You’re quite right that Dr. Craig S. Fleisher is a leading mind in measurement particularly within the public affairs space. I’m often guilty, though less and less each day given the firm I now work for, of forgetting about the unique measurement needs of the public affairs sphere. It’s an entirely different animal but no less important.
Brian,
Thanks for your note. Indeed sometimes just measuring against a client’s return on expectations is enough. I always say: measure what’s important to you and measure what counts. Not all organizations and project are created equal and measurement needs to scale appropriately. And I feel you pain on the budget front. It’s a tricky conversation.
Cheers
I’m delighted to see Terry, Fraser and Brian all weighing in with some valuable thoughts and additional Canadian measurement/research resources, as a direct result of Alan’s guest post.
And just for the record…Fraser opines to me frequently; my (repeated) request has been would he *please* share his thoughtful and valued views (garnered from a double-digit number of years of dedicated research and hands-on application, which have benefited numerous companies and NFPs) with the much wider, international audience that we are blessed to welcome at PR Conversations.
So we get the TV lady out to the bagel bakery with the Live Eye truck, and next thing we know a mom and her kids show up because they saw the first remote segment of the program at home — they arrive in time to be interviewed on-air for the third segment.
And by the end of the afternoon the bagel bakery has run out of cream cheese and smoked meat and the next morning the bagel bakery owner is at the wholesalers’s when the doors open at 6 a.m.
Good enough measurement for us.
In my forty years, almost, in PR in Canada, I’ve seen a bit of after-the-fact research, but Alan is right in his comments on budgets — one fewer digit in Canada, for all aspects of PR.
The important part of my experience yields three thoughts:
1/ Over the years, so many so-called research projects were such frauds — measure the column inches and multiply by three and then multuply by the line rate from the paper — that the nmeasurement industry has never recovered, and when most pro-research speeches and presentatins are made, they are made inward-facing, preaching to the (not yet) converted, and you can’t read the damn powerpoints anyway, nor understand the language.
2/When allocating a pr budget, we face choices like translate / adapt for Quebec, or do post-event research. Prepare a clip for TV, or do post-event research. Supply decent photos, or do post event research. Include the praries or skip Manitoba and Saskatchewan, and do post event research.
3/ And when there is money in the budget for post event research, I’ve always been tempted to turn my head around from back over my shoulder, look ahead, and spend that research money doing some other pro-active client-benefitting PR project.
For those reading this and hoping to learn who provides good PR research, I suggest taking a look at Echo Research — London, Paris and New York. Run by a Canadian, by the way. Graduate of Mount St. Vincent. Try http://www.echoresearch.com
BAK
Judy, I o’pine for you.
Alan, allow me to add some names to your made in Canada measurement influencers list:
Craig Fleisher, University of Windsor. If there a measurement guru in Canada, it’s Craig. See work with US and Canadian public affairs groups.
Sherry Devereaux Ferguson, University of Ottawa. Authority on use of content analysis. Key book: Mastering the Public Opinion Challenge.
The late Ron Pearson, Mount St. Vincent University. Just because he is the greatest PR scholar Canada doesn’t know.
Elaine Dixon, when at Mount Royal College did work on intangible assets.
—————-
Terry, I have interest in your research at Syracuse. Findings similar to many studies, it would seem. I would suggest that in your comment “The general consensus among the managers was that public relations does offer value to the organization but what limits its progress and acceptance is measurement and evaluation (along with practitioners general lack of business knowledge and assertiveness)” what’s important is in the parenthesis. And I would further suggest that “general lack of business knowledge” is too general a term and perhaps misleading. Business knowledge comes in many shapes and forms, and many/most practitioners do not work for a “business.” I would substitute lack of strategic management knowledge for business knowledge, based on my experience with Heads of PR/C functions. Benita Steyn has commented more fully on strategic management in a previous post on this blog. (See her excellent chapter in The Future of Excellence of Public Relations and Communication Management.) This term is more specific, and closer to the real issue. It takes a very “aggressive” Head to first ‘see’ (most processes are diffused with multiple actors) and second to engage themselves in a strategic management process (steps including strategy formulation; strategy implementation; change management; issues management; relationship management; emergent strategy; strategy re-formation; relationship management; change management; strategy realization; change actualization; re;lationship management; strategy formulation; etc. – at both the corporate and business unit levels.) All organizations have such a process. Whether it is articulated, organized and controlled is another question.
Measurement and evaluation are merely by-products of an engaged Head.
“Aggressive” takes us into the realm of PR/C Head as Leader or Manager (defined as maintaining the status quo). Again, in my consulting experience, the Leader role focuses on engagement in the organization’s strategic management process while the Manager role involves focus on communication and something called PR/Communication strategy (which is separate – and mostly divorced from (except for “tying to organizational goals”) – from any other “strategy”). I presented some research on Leader vs Manager at the 2004 International Public Relations Research Conference.
I’ve come to the realization that there is no such thing as communication strategy. If the PR/C Head is engaged (even facilitating) the very diffused strategic management process, then communication plans, programs and products – do as well as say communications – are simply tactical. If the Head is not engaged, then there is the invention of strategic communication plans – and some kind of parallel universe created based on high-sounding and arcane concepts like employee engagement and reputation management (for both of which in the practitioner literature there is no agreed to definition). My views are articulated in a number of Melcrum’s Strategic Communication Management articles. Benita, you can enter anywhere here!!!
Terry, both strategic management and “agreesive” require further research. I hope this is an area that your graduate program will be exploring at McMaster University in Hamilton.
Thanks Terry,
Completely agree with you. I’ve always felt, as a recovering flacktitioner myself, that the industry needs to be held to higher account and research and measurement (they are one in the same in my view) have a huge role to play in that. Communications is as much a measurement science, not strictly and art and it needs to be approached and managed as such.
Congats again on getting your program up and running. I’m convinced it will play a pivotal role in driving the agenda of a much-needed and more rigorous grad-level communications education in Canada.
Into month three here at Hill & Knowlton Canada and enjoying it.
See you around the event circuit. 🙂
Alan,
What a thoroughly enjoyable perspective…almost as entertaining and insightful as your master’s thesis from York/Ryerson. I am in complete agreement with the notion that without measurement, our profession will continue to be sidelined by the C-Suite as a sufficient but not necessary “must have”.
And measurement/research needs to be incorporated at all levels of education, not just at the graduate level where you would expect it to be, but should in our college diplomas and post-graduate certificates as well as our undergraduate programs (Mount St. Vincent and Mount Royal College).
In our newly minted Master of Communication Management program we certainly have a stand-alone research course, where both academic and applied research methods are explored, taught and evaluated — with the students having to complete a specific research project for the course. As well, more than 60 percent of the courses have as their final papers, a specific research component (whether qualitative, quantitative or mix-methods).
In a two-year project completed with my students in the Syracuse Masters of Communications Management program, we interviewed close to 200 senior managers/C-Suite members on the barrier to public relations becoming a recognized management function. The general consensus among the managers was that public relations does offer value to the organization but what limits its progress and acceptance is measurement and evaluation (along with practitioners general lack of business knowledge and assertiveness).
So congrats on your invited submission to the prconversations and congrats on your new gig…when did you start at H&K?
Terry