Balancing Your Metrics Portfolio

How do you measure the immeasurable? Things like human capital and human resources? Let’s be real, we want to know if we are properly investing in the right practices and the right people. But, the big question is how do we go about doing so?

Kevin Desouza, the Associate Dean for Research at the College of Public Service and Community Solutions at Arizona State OSD, discusses this exact issue, in a recent report titled Creating a Balanced Portfolio for Information Technology Metrics. The report was published by the IBM Center for the Business of Government and outlines some of Desouza’s major findings after interviewing two dozen well-practiced Chief Information Officers (CIOs).

Desouza sat down with Chris Dorobek on the DorobekINSIDER program to discuss the need to continue to research and innovate new Information Technology (IT) metrics, in order to better utilize the data collected to measure performance.

Government Measurements – Already In Progress

The government already possesses the ability to collect data. That data can in turn hold policy and government decision-makers accountable to their agency’s or program’s overall mission. Desouza explains that “we use a lot more data to manage our infrastructure. All of the IT applications, the hardware, and communication networks [provide] a lot of data that is leveraged to actually keep our IT environment performing well.” However, this really only applies to being effective at providing the appropriate data to manage infrastructure, not humans.

Here are two different scenarios to consider:

Scenario 1: You are standing in front of a building and are asked to look around it. Afterwards, you are asked to rate the building. This task does not seem puzzling because there are shared objectives one looks for in a “good” building (i.e. is the paint holding up, do the floors squeak, are the stairs stable, etc.).

Scenario 2: You are standing in front of stranger and are asked to rate emotions. This is not only awkward but also a bit uncomfortable. And aside from all of that, how would you even begin? Here, there are limitless possibilities on how to measure, rate, and evaluate something as abstract as emotions (personal preferences aside).

Room For Improvement

The paper’s foreword provides the simplest understanding of where we need to maintain our focus moving forward.

“Metrics on non-technical elements…such as innovation capacity of the IT department and the health of the overall IT organization — are in earlier stages of development. These metrics are critical for CIOs to effectively manage their IT departments, and to convey the strategic value of IT capabilities for attaining agency-wide objectives.”

There is one message here that needs to be made very clear. Desouza advised the government to set clear goals before selecting specific metrics. He stressed the vitality of “being very clear on what goals you are trying to achieve and the timeline under which you would like to achieve them by,” to help properly “decide the right performance indicators and the right data indicators to actually go out and collect and analyze.”

For example, Desouza recounts a reoccurring issue he heard among a number of CIOs. He explained that you do not want to potentially choose an indicator to gauge the responsiveness and the level of service your CIO’s help desk provides only to find out that “while the metric was viewed as being good, it actually was hiding an underlying issue that needed to be resolved.”

So although there will undoubtedly be a lot of trial and error during this process, Desouza reminds us that these efforts are meant to help “manage IT more strategically.” After all, Apple wasn’t built in a day!