Reliance on rear-view metrics is the root cause of many crises and has led to more than one forced investigation and/or resignation. Knowing when to recognize this form of information and how to replace it with more accurate sources can help you avoid this kind of disaster.
What are rear-view metrics?
Similar to a rear-view mirror in a car, a rear-view metric is looking backwards. In government, these often take the form of a regularly scheduled presentation including charts and a slide deck, summarizing how a particular program is doing. Sometimes a consultant or an internal team creates these reports from their own data sources, and sometimes they are compilations from many other data sources (such as a federal compilation of state-level data submissions).
Virtually all reporting in government today is rear-view reporting, and it’s often explicitly procured through a project management team and system.
Why are rear-view metrics dangerous?
Rear-view metrics are problematic for two reasons.
For one, they’re old. They are sometimes as much as years out of date, though even days or weeks of missing data can fundamentally change the decision you’d make.
Second, and usually more importantly, they’re manipulated by other people. That number you are looking at in a slide deck, or that progress report lit up in green, was generated by human beings — likely a series of humans who didn’t necessarily communicate with one another — and is therefore subject to interpretation. I am not suggesting that your slide decks are full of malicious lies; rather, the individuals contributing to the reporting may not have the context for how their contributions are being used, and people further down the line often lack the context to understand the implications of their own interpretations and manipulations.
In the end, it doesn’t matter if everyone in the reporting chain has the best of intentions: If you’re making decisions or passing along inaccurate information, you are at tremendous risk of not knowing about problems early enough to prevent a crisis (whether that’s a slightly delayed product launch or a literal disaster).
How can you make rear-view metrics more real-time?
Trace the data you’re relying on all the way back to the source. Look for novel ways to generate real-time dashboards of data using source data. Use an IT-approved, off-the-shelf collaborative work management tool that can shuttle data between even the most ancient legacy databases into a web-based sheet; this step shouldn’t require a big investment of dollars or technical expertise.
For most purposes, an information sync once a day or even once a week might be completely sufficient; if you’ve been operating off of manually generated slide deck up until now, you definitely don’t need up-to-the-second data feeds. The key is that you’re automatically, consistently generating your reporting from your data — humans aren’t manually interpreting or massaging the message anymore.
When you manipulate data, such as deciding if a number is “red” or “green,” document it clearly, and run it by the experts in the original data. They may point out critical distinctions, overlaps, or other consideration that make your seemingly simple summary totally wrong.
Marina Nitze, co-author of Hack Your Bureaucracy, is currently a partner at Layer Aleph, a crisis engineering firm that specializes in restoring complex software systems to service. Marina is also a fellow at New America’s New Practice Lab, where she works on improving America’s foster care system through the Resource Family Working Group and Child Welfare Playbook. Marina was the Chief Technology Officer of the U.S. Department of Veterans Affairs under President Obama, after serving as a Senior Advisor on technology in the Obama White House and as the first Entrepreneur-in-Residence at the U.S. Department of Education.
Leave a Reply
You must be logged in to post a comment.