The Oversimplification Trap

Unless you’ve been living under a rock, you have no doubt heard the term big data, especially if you’re a GovLoop reader. As more and more processes move online, agencies have access to millions and millions of data points they didn’t before. While this presents myriad opportunities for improving government, the challenges of interpreting massive data sets demand a higher level of analytical skills, as well as a healthy respect for the unpredictability of human nature. In the absence of these, agencies can fall into an ‘oversimplification trap’ –  attempting to manage a complex process based on a few easily captured, but superficially important, metrics.

One example of such a trap is how GSA used a particular metric from its online contracting tool, eOffer. eOffer is the acquisition system that facilitates contract actions for GSA Schedule holders. With more than 15,000 contractors on the GSA Schedules, you can imagine how much information can be gleaned from this system. Keep in mind that ALL types of modifications are run through eOffer, from simple deletions to much more complex actions such as novations. GSA chose to manage its contract modification process, in part, based on turn time, or the number of days it takes a contracting specialist to complete a contract action.

Once management began to focus intently on turn time to improve the modification process, an interesting thing happened, getting a modification awarded became significantly more difficult. Rather than potentially damage their center’s metrics, some contracting professionals rejected modification requests that would take too long to negotiate. Some of my colleagues and I have even had price reduction modifications rejected because they sat in the queue for too long. The use of rejections gave contracts personnel the power to deliver the expected results, but also made it considerably more difficult for GSA’s management to understand how long it actually takes to complete different types of modifications.

Don’t get me wrong, I truly believe GSA wants to improve the modification process. I attended an industry conference last fall that included many of the top people from GSA. In one session, participants were asked to provide feedback about some of the toughest issues facing GSA schedule contractors. The issue eliciting the most discussion by far was the modification process. A GSA official seemed shocked that industry was still having problems with modifications because the processing time has never been faster. While that may be the case, who determined that faster is always better, and who is tracking the modifications that are rejected for being too time-consuming?

This type of problem can happen when we try to boil any exceedingly complex system down to a few metrics. That is why it is critical not to do data analysis in a vacuum. It allows users to give you the answers you want instead of the information you need with you none the wiser. Ignoring the impact of human behavior on any process can easily lead into an oversimplification trap. Getting honest, real-time feedback from the stakeholders can help lead you back out, as long as you are willing to listen.

Jennifer Aubel is part of the GovLoop Featured Blogger program, where we feature blog posts by government voices from all across the country (and world!). To see more Featured Blogger posts, click here.