Data, Data, Everywhere

We’re producing more data – and increasingly more unstructured data – than ever before. This seemingly endless flow of information from a wide variety of sources has facilitated valuable insight into previously unknown areas and has unlocked significant economic potential. But for government, as well as other sectors, data is only good if it can help achieve your objectives. How does data apply to your agency’s mission? And how can your agency apply data and make good use of it?

At the recent Cloudera Federal Forum, Mike Olson, Chief Strategy Officer and Co-Founder of Cloudera, delved into these topics and spoke about how “pervasive analytics” driven by an enterprise data hub (EDH) can effectively meet the needs of your agency. As expected, the issue of security was discussed, as well as some case studies from organizations that have seen the benefits of big data analytics.

From Past To Present 

In the past, organizations could not have foreseen the current demand to ingest complex, unstructured data. Frameworks were limited and only included transactional, relational data that didn’t have analysis capabilities across a broad range of data, Olson said. Furthermore, administration and security were fragmented, as new siloes would often be built onto existing infrastructure.

As a result, it was very difficult to act on data. It could take months for an analyst to get the data he wanted, forcing a “gut decision” instead of a “data decision” that could likely lead to a less successful mission.

Today, this environment has changed significantly. Agencies that deploy an EDH now have one place to store and readily access all of their data for as long as desired. And it is very cost-effective, so you don’t get rid of information that may not be useful now, but can be potentially valuable in the future.

Schema-on-Need

This is the idea of “schema-on-need,” Olson explained, that allows the flexibility to remain agnostic with your dataset early on, and define a schema when it’s the right time for you. There are also new query methods such as graph traversal – seeing who’s connected to whom – and machine learning – having software deduce patterns and spot outliers.

What is really valuable, said Olson, is having a place to land data and capture it all in one place and without loss of its native fidelity. By unifying it, you therefore unify the framework as well as the security and administration of the data. This advocates openness and also promotes operational as well as regulatory efficiency.

Know More with Analytics

An EDH allows you take in unstructured data and facilitate it with structured, relational data and ask questions you couldn’t have asked before. You can use analytics to reason effectively from varied sources, said Olson. These capabilities empower the user to effectively utilize and explore data possibilities.

Furthermore, analytics allows you to gain real-time insight. As David Bennett, Director, Implementation and Sustainment Center, Defense Information Systems Agency (DISA), mentioned earlier in the conference, “If you’re looking five minutes behind, you’re already six minutes late.”

Is it Secure?

As in any discussion about big data, security is a major concern. The capabilities of big data are vast, but no one will bite if you can’t guarantee security, said Olson. With a single shared data management platform, it’s important for personnel to share information and insights, but only what they’re entitled to share. Data governance allows you to capture who touched what data, and what they did with it. In addition, continuous data streaming watches for abnormalities based on historical trends and baselines. With these assurances, you can be confident that your data is secure.

Real-World Examples

These capabilities have been used for veteran communications and mental health. By utilizing analytics over social media platforms, signs of stress can be identified from unstructured data capture (tweets ,Facebook posts, etc.) and acted upon accordingly. Furthermore, healthcare IT leader Cerner, has saved hundreds of lives by utilizing Cloudera big data analytics for disease intervention. The insight pulled from disparate medical and personal data creates an exceptional, personalized, and efficient patient-doctor relationship.

Yesterday’s limited, structured data storage has made way for today’s unlimited disparate data storage through database management systems such as an EDH built with Cloudera. By managing, analyzing, and acting on big data, you can effectively change agency outcomes.