When used properly, statistics can be very informative. However, some statistics are meaningless, and some are dishonest. Interpreting statistics requires some technical knowledge, and most people do not have the basic training to know how to read statistics and to take them with a grain of salt. Statistics are particularly misleading with regard to the early days of any phenomenon because percentages are distorted by the small base: moving from 1 to 8 readers/users/etc. is an 800% increase. Of course, eight out of six billion is a drop in the bucket…unless those are the eight most important people on Earth for your purposes, like the G8 heads of state and government. Maybe the hotline between the White House and the Kremlin will be replaced by direct Twitter conversations.
The various measurements put forward to evaluate ROI for communications illustrate how badly statistics can be used. Too often such statistics are an activity report, not an analysis of impact. At the end of the day, impact is what matters. Or worse, they measure totally irrelevant things.
A heated debate has just taken place over at Six Pixels of Separation about HubSpot‘s report “State of the Twittersphere 2009“. To me the quoted statistics in this article don’t tell me whether Twitter is useful for me or not, because they say nothing about impact. The definitions of the segments is tautological since everyone is defined by their relationship with Twitter rather than their relationship to me (or you or a brand…).
Full disclosure: I am one of those people mentioned who signed up for an account and has yet to figure out how to use it (and was put off by the fact that it won’t accept my mobile phone number for remote posting). I’ll get around to it, just like I did for LinkedIn/Viadeo, Facebook, Skype, etc. But in each case, I had to find the compelling reason or the lull in my schedule to invest in the initial learning curve.
Twitter got my attention last week when I was reading the International Herald Tribune (which I usually only have time for when I am in an airplane or airport lounge). Entitled “Washington Taps Into a Potent New Force in Diplomacy“, the article explains that the White House asked Twitter to delay some routine maintenance to ensure continued service to protesters in Iran who have been using Twitter to coordinate their activities. All of the articles on events in Iran — and there were quite a few — mentioned the role Twitter was playing.
“Wow,” I thought. “That’s impact! That’s useful. That makes sense to me.” (More disclosure: my degrees are in political science.) So I finally understand the usefulness of Twitter. This turn of events is likely to encourage me to finally domesticate Twitter myself, even though I am not planning any major social/political movements any time soon.
I was wrong. Nigel Hawkes is no longer with The Times, but is now director of Straight Statistics. Moreover, I’d like to recommend a lecture, which examines how all markets are based on trust. That explains how confidence is everything. Yet investors must take risks. It examines the role of trust in commerce and power while setting out some early thoughts on the Theory of Influence. It is by Professor Michael Mainelli of Gresham College. Enjoy!
http://www.gresham.ac.uk/event.asp?PageId=45&EventId=838
Great post, Kristen! Journalist Nigel Hawkes of The Times recently announced the launch of Straight Statistics, an idea of Lord Lipsey, a former deputy editor of The Times. Supported by the Nuffield Foundation, it exists to highlight abuses and celebrate successes in the use of statistics.
The PR industry has much to be ashamed on this issue. Some of us (many of us?) have knowingly manipulated statistics. We have done so when announcing either financial results or those of surveys. But the reason PRs and their clients have got away with it is because of the gullibility of both audiences and the mediation services they rely on, such as journalists.
There is also another problem: weak interpretation of good data. Take the Edelman Trust Survey. It is one of the very best surveys out there. It is professionally conducted and done so annually. It provides a wealth of honestly presented data. But most commentators – including Edelman itself – have not interrogated the data. We are mostly meant to treat the results as if they were transparent-proof points of reality (as fact).
As to Twitter, my wife working as a journalist from Zurich relied on it for information during the news blackout in Iran. But she – a Swiss – could only read it in English, as was the case for most people who were not Farsi-speakers. I maintain, as a Twitter fan, that its real impact inside of Iran was marginal. But, I admit, I have no proof.
Here’s some links for anybody wanting more:
http://www.timesonline.co.uk/tol/comment/columnists/guest_contributors/article6523495.ece
http://straightstatistics.org
http://paulseaman.eu/2009/01/would-you-trust-a-trust-survey/
http://livingissues.com/2008/04/14/bogus-consumer-research-and-pr/
Markus — Once I actually start posting anything on Twitter, I will happily share my Twitter name.
KD — I agree that bad statistics do us all a disservice. But clearly just saying so isn’t getting rid of the practice. I suspect that part of the problem is statistical illiteracy on the part of many communicators (who will tell you that they are in communications because they don’t like numbers). In my view, this also reflects a wider technical illiteracy in our society…but that is another blog post altogether.
you are SOO right. The folks out there that are inventing “ROI” metrics based on Ad Equivalency and similar nonsense are not doing our profession any favors.
The real ROI measures are based on solid business results, either money saved or revenue brought in.
Kristen,
spot-on. Like all other social web applications twitter is not useful in and by itself, except maybe for the aspect that it enables or simplifies communication for everybody. twitter’s role as a channel to distribute information about and organize the post-election protests in Iran is impressive and shows – convincingly, I might add – that it is something not to be neglected by PR.
As for personal usage: A good first step would be to let us know your name on twitter so that we can follow you 😉