How to Move Beyond DIY Data Initiatives

Sooner or later, every organization needs to realize that data-driven decision-making cannot be a do-it-yourself project.

The DIY approach might have worked in the beginning. Many agencies began their journey toward data-driven decision-making when a data enthusiast downloaded some analytical tools and demonstrated the kind of insights that could be gained.

That worked well initially, but over time organizations found themselves with countless data silos, each with its own objectives and toolsets.

As agencies move toward adopting data-driven decision-making as a strategic initiative, they need to take a different approach, simplifying how they store and manage that data while still providing flexibility in the choice of analytic tools.

That’s the idea behind a data fabric, said Stephen Moore, Chief Technology Officer at AlphaSix, a data management company.

Future-Proof Data Management

A data fabric provides a unified platform to store and secure your data so that it is accessible to groups across the organization. And that platform essentially is decoupled from specific analytic tools, which means an organization’s toolset can change as requirements change, Moore said.

“If today you’re using a visualization tool, for example, and two or three years down the road you decide there’s a new tool that you’d like to use, you can swap out the tool without having to change the underlying platform,” he said.

Another critical piece is the cloud, which brings additional flexibility and scalability to data initiatives.

“If you have a sudden surge in data, and you need to scale up more resources, you can do that with just a few clicks of the button,” Moore said. “Or if you want to test a new service, you can quickly spin it up, see if it helps, and then spin it down.”

A Data Fabric in Action

In one case, AlphaSix worked with an agency to build a solution for security analytics.

The goal was to capture log data from systems throughout the organization, including desktop computers, servers, applications, security appliances and countless other devices, and provide tools for analyzing that data and getting insights into the security of their environment.

The solution was built largely on open source software. It included:
• Hadoop, a distributed file system that is designed to handle large volumes of data
• Elasticsearch, a search and analytics engine that can be used both to run queries and to build dashboards
• Spark, which provides algorithms and tools for machine learning

AlphaSix also provided tools for automated data preparation, which shortens the time it takes to get from ingesting data to analyzing it.

You Don’t Have To Do It Yourself

This is the kind of approach that agencies need to take to move beyond DIY efforts and achieve the goal of data-driven decision-making. This is the kind of work that AlphaSix specializes in.

“We understand how to build these systems, to make them future-proof, and scalable, and give you a lot more flexibility for what you can do with the data,” Moore said.

This article is an excerpt from GovLoop’s guide “Your Field Notes for Data-Driven Decision-Making in Government: Case Studies on Work Culture, Equity and More.”