10 Elements of Proper Data Management

In today’s world, data powers economic growth, enables the shared economy, creates new kinds of social interactions and delivers valuable business intelligence to organizations – all while driving increased productivity and efficiency.

Analysis of data presents the opportunity to tackle some of the most challenging issues facing our society. It’s data that helps us understand how to improve delivery of water to remote locations, provide Internet access, create completely automated vehicles, and improve the treatment and diagnosis of diseases.

As more organizations learn how to leverage data, it’s becoming increasingly clear that we are on the brink of remarkable innovations. And for government today, the challenge is finding the right solutions and infrastructures to not only capitalize on the data it has today, but to build a foundation for future growth.

But what does government need to do to capture information and turn it into actionable insights?

One of the keys is being able to centrally manage data. Technology has developed in such a way that the concept of centrally storing data for all walks of analytics and use, regardless of file type or format, is now a possibility. Today, this can be done in a way that is cost-effective, while avoiding the pitfalls of data replication and data movement that plagued previous designs. Many public sector organizations have now realized these advantages and have adopted this technical architecture, which is called an enterprise data hub (EDH).

With an EDH, data can be stored in its original fidelity, integrated with existing infrastructures, and managed as a flexible, single source of detailed information that can drive a wide range and variety of workloads – such as batch processing, interactive SQL, enterprise search and advanced analytics.

An EDH allows organizations to centrally store this data with the proper data protection, governance, security and management needs required for government. Powered by Apache HadoopTM, an open solution for storage and computing, agencies can now capitalize on data with the cost-effective benefits of open source. This includes flexible, scalable deployments and access to both commercial vendors and developer communities for support. As OSS typically aligns IT expenditures more with the application and user, and less on the software infrastructure and plumbing, agencies can maximize their IT dollars..

Although not an exhaustive list, below I’ve listed ten keys to implementing a successful EDH architecture. These ten items can help you build the foundation of an EDH to start to transform the business of government.

1. Leadership support: Without strong leadership and support from management to create a data-driven culture, organizations will struggle to capitalize on data.
2. Accessibility of information: Data should be accessible, and information should be easy to access by those who need access to information.
3. Integrated systems: Data should not be distributed among separate and disparate data systems. Information should be housed in one spot.
4. Leverage existing infrastructure: Agencies don’t always have to reinvent the wheel; they can also work to leverage existing infrastructures and applications.
5. Run operational and advanced analytics: Once data is collected and housed in a single location, agencies can start to run advanced analytics on their information. This can provide valuable insights to leaders and help them drive better business decisions.
6. Data type and file agnostic: Agencies are creating various kinds of data, and it’s important that their infrastructure is able to search across data files and understand how to leverage insights, regardless if it is text, email, and unstructured or structured data.
7. Easily search for information: Employees must be able to search for information and find what they need quickly. Any EDH must be equipped with the ability to search to improve efficiency and find information quickly.
8. Robust governance policies: Data governance is essential to assure proper data management; organizations should look at deploying the right processes and systems to preserve data quality.
9. Safe and secure: Agencies should have confidence that the information that they have can be secured and kept private.
10. Scalable solutions: As organizations continue to create more and more data, it is important that we develop scalable solutions, so that we can prepare to meet the needs of tomorrow by investing properly today.

These ten themes create a framework for proper data management. Within this framework, there are many ways to make data more accessible and make improved decisions. So how can you begin? Like most things, it can start by asking the right questions.

• What is our current data architecture? How can it be improved?
• Do we have a data governance policy one? How often do we review it?
• What kind of challenges does our organization face to leverage data?
• How do we encourage support from all teams? What kind of support and commitment is needed from each team?
• Are we laying a foundation for future data initiatives? How do we scale as needed?
• What kind of data will we need to extract in the future? How can we think differently about the data we already have?

These questions are just the start of your journey to leveraging data in new and transformative ways. Use this question set as a means to start a great discussion on how to fully leverage data at your agency, and be sure to check back to GovLoop as we explore the questions above.

Cloudera is revolutionizing enterprise data management with the first unified Platform for Big Data: The Enterprise Data Hub. Cloudera offers enterprises one place to store, process and analyze all their data, empowering them to extend the value of existing investments while enabling fundamental new ways to derive value from their data. Learn more here.