Event Streaming: The Future of Data Management

For agencies, data is not just the engine for innovation — it is the engine for everything. Agencies that use data effectively can better collaborate with one another, deliver stronger services to citizens and achieve more mission wins.

Despite this, many agencies have taken a passive approach to managing their data, resulting in disconnected data silos and outdated applications and practices.

Event streaming can help agencies handle these issues with active data management and sharing. GovLoop spoke with Will LaForest, Public Sector Chief Technology Officer (CTO) and Jason Schick, General Manager for U.S. Public Sector at Confluent, an event streaming platform provider, to understand what this approach is and how it works.

LaForest and Schick shared the following three best practice tips for integrating event streaming into agencies’ operations.

1. Change the Data Culture

Traditionally, agencies have processed data in batches, which means any data set is essentially a snapshot in time. Event streaming, on the other hand, treats data as a never-ending stream of events, with data being updated in real time as changes occur.

“The idea is simple: every time there is a change to a source database, that change is distributed to everyone who cares,” LaForest said.

Gradually, event streaming improves how agencies analyze their data, boosting the productivity their workforces derive from this information.

2. Share the Wealth

Ultimately, agencies want simple and rapid access to all the data available in their various environments at any time. Connecting every form of data is crucial for agencies, whether it is historical data from repositories, such as legacy databases, or new, real-time data. Event streaming platforms ensure all data sources are constantly available and up-to-date. After that, the platforms enable agency employees to engage with any data they need.

“You remove the institutional friction of sharing data: negotiating with other organizations and finding out what the schema of the data is,” Schick said.

Overall, event streaming platforms boost cooperation inside agencies while fostering more collaboration between agencies, their staffs and the outside world. This collaboration stems from the fact all parties are working with the same data.

3. Act Proactively

Event streaming can become a building block for agencies’ futures, according to Schick. Agencies that have readily available data can anticipate and prepare for problems, such as natural disasters, more intelligently. The more informed agencies are, the wiser their decisions can be.

“You can recognize things faster and put that data into motion to produce a lot of benefits from everything to safety and defense to improved voter registration and transportation,” Schick said.

This article is an excerpt from GovLoop’s recent guide, “Reinventing Government: 20 Innovations for 2020.” Download the full guide here.