The government of Canada has recently put together a group to come up with techniques and best practices to measure social media in the public sector.
We’d like to see what other governments are doing to measure their use of social media–whether it’s quantitative or qualitative and what kind of performance measurement frameworks are being developed or used.
So, we’re checking out presentations like the one Andrew Krzmarzick posted last December https://www.govloop.com/profiles/blogs/measuring-social-media-and (thanks, Andrew) and thinking about topics like:
- how to analyze traffic in channels like youtube, twitter, facebook, etc.
- how to think about ‘conversations’ with key audiences and how best to engage in activities like corrective blogging’, for example
- how to measure qualities like ‘tone’ of conversation, ‘brand’, ‘reputation’ and ‘sentiment’
- how to put together a ‘return on engagement (ROE)’ framework
- how to communicate the quality of ideas, and
- how to measure how well we’ve reached key audiences.
Of course, there are many other ideas and tools available that can be addressed–depending on the objectives of what a government service, program or information project sets out to achieve.
If you have any ideas, experiences, presentations or tools that you’ve used and would like to share please let me know.
I wondered if you had thought of anything similar for internal social media use where the audience is colleagues and team workers and the communication has a rather different purpose. Or do you know of studies on the effectiveness of internal social media?
Barry
One thing I’ve suggested in the past is to look at the usage patterns of social media tools. Perform an analysis of both the type and frequency of usage.
As an example, last year I did a quick assessment comparing how two different Combatant Commanders – CENTCOM & EUCOM – made use of their Twitter presence. (*) My analysis is at the link referenced below. The image attached to this comment is just one piece of that analysis.
Examples of some of the questions answered by doing this analysis:
Question: Are we using the medium interactively?
Measurement: Comparison of original tweets (one-way) vs. Retweets/@replies (interactive)
Question: Are we listening or only talking?
Measurement: Comparison of followers vs. following numbers.
Question: Are we timely and responsive?
Measurement: Tweets over days and times. If all are in morning (as in CENTCOM example), it is a “check the box” activity and probably not timely and responsive.
This analysis was specific to Twitter, but the same principles could be applied to any other social media or Gov 2.0 presence.
Measurement: Interactively using the medium
Are we using the medium in a one-way or interactive manner? (Tweets vs. Re-tweets/@replies)
(*) Note: The analysis is over a year old. The referenced link and analysis may be radically different from their current usage patterns.
REF: Twitter Feed Comparison – EUCOM vs. CENTCOM (as of April 2009)
CDC has some pretty good metrics – check out the social media ones on right
http://www.cdc.gov/metrics/
Thanks to all who have posted this a.m.!
Bridget: Right now, we’ve not considered measuring much about internal comms since to my knowledge the Government of Canada has just seen wikis being used within departments and agencies and more collaborative platforms are just now being either considered or implemented. However, this blog thread could certainly pick up any suggestions in that internal communications space.
Bob: I think your distinction between ‘one way or interactive’ (tweets vs. RT’s) is very helpful. Thanks for the suggestion and images. I think you’re correct. Once you’ve established what you’re doing by answering the questions, you’re in a better position to figure out what you’re looking for.
And GovLoop: thanks for the CDC link. We were particularly interested in the CDC work using twitter and other SM during the H1N1 events. Very instructive lessons. I’m checking out other firstgov. usa.gov sites, too.
@Barry – I think Angelina Munaretto is the best to ask…another person in Canada is Nelly Leonidis. Finally, Laura Wesley has been writing some great stuff about performance measurement and may have some feedback on social media metrics.
@Bridget – just asked an enterprise expert to weigh in, too.
@Bob – the stuff you shared is awesome, thanks!
@Steve – I think CDC is the gold standard by a mile!
Thanks, Andrew. I can track all of those good folks down. But we’re looking for other governments’ ideas, too.
An interesting coincidence, as an experience today provides another great example.
Earlier today I saw the following Tweet and gave a tongue-in-cheek response:
Normally I’ll take the time to see if the organization I’m referring to has a Twitter ID and include them. In this case, I was a bit lazy and just posted U-Haul’s URL.
So, I was quite surprised – and impressed – when I saw @UHaul_Cares Re-Tweet my remark less than 30 minutes later.
That response demonstrates several factors:
— They are proactively monitoring social media nets for their brand. And they are not just searching by their ID, but their name and website address also.
— With today being Memorial Day, that proactive presence may be “round the clock” with dedicated monitoring, most likely in the form of automated searches.
— There is a “human in the loop” – if not then their AI responder is better than any I’ve seen. Both of their responses were personalized and in proper context.
After noting their quick response, I wrote back:
To which I received another response:
So, I’ve just shared a great example of what I consider “doing things right.” But back to the topic at hand, how do you measure that?
I’d suggest the following as an algorithm to use.
1. Periodically (daily, weekly) search the SM space for mentions of your brand or agency.
2. Develop a mechanism rate those mentions (positive, negative, neutral; 1-10; etc) as to whether they should receive a response.
3. Assess how many of those did receive a response, were noted or made it to the appropriate department.
If developed with all of your objectives in mind, the measurement tool also becomes the engagement tool. The customer service (or equivalent) department uses the same tool to identify places to engage.
If an organization does not already have the “engagement identification” tool, then it’s probably no surprise they do not know how to measure what they are not doing… 🙂
Good discussion and glad that the work is going on.
When we showcase mission-relevant, tangilble success stories – these should include value whose metrics are clear.
Agreed, Bob: “If an organization does not already have the “engagement identification” tool, then it’s probably no surprise they do not know how to measure what they are not doing… 🙂 ”
Our (GoC group) discussion began with a discussion about the requirement to put together business plans that include communications objectives that then can be applied to many channels including what (if anything) should be attempted and measured using SM.
We’re trying to look at how others set up that kind of framework and then apply relevant performance measures.
I think that’s where Greg’s points come in: determine the value of what you’re attempting in terms of the overall objectives such as service levels, program/project objectives, ‘reach’ within defined audience segments, etc. The you can set up the measurements–either quantitative or qualitative that apply. Hopefully, within that kind of context (and the thinking required to get to that point) clarity in terms of metrics should be easier to set out–that’s the theory, anyway.