Government and Big Data: Benefits and Challenges

There are hundreds of articles on the growth of big data and current uses but what might the future look like for big data in government? Can government reap the same rewards as the consumer market by using and sharing big data?

Big Data Growth

We are producing 2.5 Quintillion bytes of data every day and Forrester predicts the market will grow 12.8 percent over the next 5 years with Hadoop growing 32.9 percent annually. The highest growth is expected in the pharmaceutical and transportation industries.

Data lakes are now the big buzz but there’s already too much information to sift through. Value is coming from efficiently extracting information from all that data. Lakes and data warehouses are all growing silos within individual organizational borders. That’s why it is important that government considers how to share information between agencies when storing it.

I think we are going to see huge growth in data sharing over the next few years that will rival the initial growth of the internet. While there will remain significant growth in consumer behavior data, growth in the availability of sensor, scientific and medical research data will see huge gains.

Universities and Big Data

Many universities need to start sharing raw information for further research. I think we could see more virtual experiments carried out using shared research data and advanced analytics. This would bring about advances in medical and scientific discoveries. Government and industry-backed research should stipulate how the research data is to be shared in order to benefit and advance all research.

Will Government Be Left Behind?

The Government benefit of big data is obvious. The big data consumers would include law enforcement, education, government audits, urban planning, economic development, emergency planning, and military intelligence. Agencies can harness the same publicly-available data to correlate with government-maintained information to gain new insight and efficiency in public service.

Don’t Forget Security!

The benefit of sharing information between government and civilian industries will be enormous but not without the burden of protecting it. Big data sharing is inevitable so protecting the data at rest and in transit should be the highest priority.

Where Do We Start?

So how can government agencies take advantage of big data and sharing?

• Start looking at how civilians are using big data and what data is available.
• Look at what data would benefit current processes and how that external information could be correlated with internal data to produce value.
• How are other agencies using big data to create efficiencies and value?
• What information do you hold that could be of value to other agencies?
• What data do other agencies hold that could be of value to your agency?
• What is the mutual benefit of sharing big data between your agencies?
• What is the cost of sharing and protecting this shared data?

The tasks of using and sharing big data for government seems huge. However, the benefits and potential cost savings would outweigh the costs. Rome wasn’t built in a day and neither will government use and sharing of big data. You cannot finish building a city until you lay the first brick!

Matthew Scott Eagles  is part of the GovLoop Featured Contributor program, where we feature articles by government voices from all across the country (and world!). To see more Featured Contributor posts, click here.