The use of big data has been gaining momentum over the last few months. The term big data still has some ambiguity in how it is defined, and for government there are numerous challenges to unlock all the benefits and promise of big data for agencies. Commonly, big data is viewed as taking large volumes of data and using the data to make informed decisions within an agency. The hope is that by using more targeted, tailored and on-demand data, agencies can cut costs, reduce fraud and deliver higher level of services to citizens.
There have been countless interesting applications of using big data at all levels of government. At the local level, you commonly find stories about police departments using big data to fight crime, federal agencies synthesizing large volumes of data to improve customer service, and state agencies collect volumes of data as to administer grants, educational programs, healthcare, the list is quite extensive.
Managing all this data does not come without related challenges for agencies. Below are some of the challenges I see with big data and barriers agencies need to overcome to leverage all the potential of big data.
IT Infrastructure
The first challenge is developing the right infrastructure to hold all the data. Agencies collect enormous volumes of data, so managing the data, making sure it is accessible to the right decision makers, moves throughout the agency, is one obstacle government needs to overcome. There obviously is a cost associated here, so agencies need to be sure that up-front investment will lead to long term savings.
Personnel/Workforce
Understanding complex data is not easy, so agencies need to have employees with strong skills in statistics to help derive value from the data. Implementing new technology and having the right infrastructure is great – but you need have a solid staff who can understand the data that is being produced.
Identifying Scope
Agencies need to purely define what they want to do with the data, what the impact is, and how the data will be used to drive decisions. Without scoping out the project plan, and understand the end goal of data, the data will not be fully leveraged for an agency.
As always, I’d love to hear your perspectives on what is happening within your agency and how you are leveraging big data.
What have been some of your challenges with big data? How would you define Big Data?
 |
Oracle offers an optimized and fully integrated stack of business hardware and software systems that helps organizations overcome complexity and unleash innovation.. Check out their Optimize with Oracle group on GovLoop as well as the Technology Sub-Community of which they are a council member. |
I think your spot-on with those challenges. We’re also going to need some larger screen areas to better visualize all that data, at least for certain functions.