Big Data, Big Deal

Yes, big data is a big deal which has contributed to the big hype.

But, behind the hype there’s a simple story. For decades, organizations have been making decisions based on transactional data stored in relational databases. Today we extend beyond that critical data, however to leverage a potential treasure trove of less structured data: weblogs, social media, email, sensors, and photographs that can be mined for useful information.

Big data is like traditional data in many ways: It must be captured, stored, organized, and analyzed. However, results of the analysis need to be integrated into established processes and influence how the organization operates and performs.

The Federal government has contributed to making big data a big deal by announcing the “Big Data” Initiative on March 29, 2012. The initiative encourages government organizations to advance state-of-the-art core technologies needed to collect, store, preserve, manage, analyze, and share huge quantities of data.

But, what can government organizations do about big data to comply with the “Big Data” Initiative?

Previously, there really wasn’t a cost-effective way to capture and store big data which would have made compliance with the “Big Data” Initiative extremely challenging. However, the good news is that new technologies are emerging that enable organizations to get their arms around these vast quantities of unstruc­tured data, making the prospect of gaining insight both feasible and cost-effective. Key enablers for analyzing big data are tools for statistical and advanced analysis. These tools must be able to work with distributed data to perform analysis regardless of where the data resides, to scale as big data volumes grow, to deliver response times driven by changes in behavior, and to automate decisions based on analytical models.

Managing these vast quantities of data is only half the battle. Organizations that hope to truly benefit from big data must also integrate these new types of information with traditional organization data, and fit the insight they glean into their existing business processes and operations.

Those organizations, specifically in the private sector that are truly leveraging big data understand that they must analyze their new sources of information within the context of the bigger picture. This perspective requires the integration of big data with traditional data to gain a 360-degree view across the organization. This approach offers a more complete understanding of the organization and it also builds upon existing IT architectures instead of replacing them.

Where should organizations start?

Organizations are advised to take a phased approach to leveraging big data. Initial projects should be small in scope: identify one set of desired data, capture it, explore new data-management techniques, and determine integration points with existing data. Starting with pilot projects and building on successes will help organizations realize the benefits of leveraging big data with minimal disruption to operations.

The adoption of “state-of-the-art core technologies” highlighted in the “Big Data” Initiative designed to specifically address big data will enable organizations to analyze new types of information that hadn’t been feasible to analyze before. By adding this information to the mix, organizations can leverage the best of structured and unstructured data to find new solutions to challenges, optimize information and service delivery to citizens, and significantly increase employee produc­tivity through streamlined processes.

There is potentially big payoff from big data.

The recent Winter Corporation report, “Big Data: Big Business Opportunities, Requirements and Approach” provides additional perspective on solutions and available technologies to help organizations address the big data challenge.

Additionally attend the “Big Data Summit for The Federal Government” to be held Thursday, May 10^th at 8:30 a.m. in Reston, VA for a discussion on how to acquire, organize, and analyze big data alongside traditional enterprise data to enhance decisions and operational value. Click here for more information or to register. Alternatively, reference the following link:

http://www.oracle.com/us/dm/h2fy11/65918-nafm11032890mpp009c003-oem-1583337.html