Predictive Analytics and Insights Discovery Were Made For Each Other

Every organization wants to take full advantage of their data—to make more insightful decisions; to be more efficient; to gain more market share. But they also want to go beyond making projections, to drilling deeper into cause and effect. What would be possible if your organization could predict future occurrences? A new generation of advanced analytics—high-level diagnostic, predictive, and prescriptive—can now provide that opportunity.

Thanks to breakthroughs in data science provided by large multi-dimensional datasets, predictive analytics is transitioning from hindsight to foresight and insight, making a game-changing leap over current practices. These new capabilities can move organizations that have already laid the groundwork in Artificial Intelligence (AI) and data mining to a new level of maturity.

Levels of Analytics Maturity in Data-Driven Applications

While descriptive analytics can show what has happened in the past, or what is happening now, they don’t explain why things happen. That’s the role of diagnostic analytics, which looks for

relationships in the data that can provide clues to causes and effects. Diagnostic analytics works with the foundational descriptive analytics to provide the next level of understanding: insight. Insight then delivers prescriptive power—the power to optimize (to prescribe) what happens.

How are prescriptive analytics different from predictive? While large collections of data and information assets are hugely helpful to analysts, high-level diagnostics can now go a step further. They can map out the complex connections and patterns in the data, showing the impact of various forces on others. For example, these diagnostics might reveal a host of factors that in some way or another have led to unexpectedly high attrition in a particular division of a company. The diagnostic would show which factors had the most impact and which had the least impact. Hence, we can do more than predict an outcome based on a set of observed circumstances—that would be predictive. We can now learn (from insights encoded in big data collections) which modifications to the circumstances can lead to better (hopefully, optimal) outcomes—that’s prescriptive.

These insights can even include critical factors that analysts hadn’t thought to look for. For example, they could uncover that an increase in mechanical failures of a pump has less to do with a change in the operating environment than with certain members of the maintenance team who all shadowed the same instructor.

With predictive and prescriptive analytics, we can look around corners and see how the scenario created in the diagnostic might change if we take a particular action. Predictive and prescriptive analytics go far beyond manually extrapolating data from spreadsheets, to making projections and to driving insights-driven decisions. These advanced analytics capabilities rely on models that “think” through any number of possible scenarios and assign each one a likelihood of occurrence. The analytics do not actually predict the future – they are not fortune tellers. They merely provide the probability that events will unfold in a certain way, based on data about how similar events have unfolded in the past.

The outputs of predictive analytics are typically expressed as likelihoods—such as 85% probability of Outcome A, 10% Outcome B, and 5% Outcome C. These likelihoods are based entirely on data from events that have occurred in the past (i.e., historical training data). Prescriptive analytics go even further—by learning the responses of a system to various causal factors, we can now assign probabilities to different outcomes under new conditions, circumstances, and optional treatments that we might decide to implement.

When weather forecasters show a map of potential paths of a hurricane, they’re using predictive models that combine real-time information with historical data about how previous hurricanes have behaved (i.e., using computational models whose features and parameters have been validated by historical hurricane data). The more data that’s available—whether for hurricane paths, or talent retention, or pump failures—the more accurate the models.

The Power of Combining Predictive and Prescriptive Analytics

With the combined power of predictive analytics and prescriptive analytics (derived from insights discovery in big data), we can explore more insightful data-driven “what-if” decision scenarios. This type of “what-if analysis” can foresee the ripple effect of various alternative decisions that adjust different elements in our operations. When we use prescriptive analytics to turn the dials of those elements—either singly or in combination—we can see the implications across the entire landscape.

The outputs of prescriptive analytics are particularly helpful in assessing risk, such as when making a strategic decision, or in deciding whether to make cuts or investments. Predictive analytics evaluate the risk of current conditions continuing on their current path, expressing the risk as a mathematical probability. Prescriptive analytics evaluate risk in new scenarios that unfold based upon different decisions, treatments, and options that we might choose to take. This is powerful insight when balancing priorities and considering tradeoffs.

Moving from hindsight to foresight with predictive analytics is a big jump. Moving from there to prescriptive analytics, based on insights discovery, is a game-changer (perhaps a “hockey stick moment”, for a significantly better end-state). Organizations who have already laid the necessary groundwork for analytics, are now ready to move to that next level of advanced analytics. Even those without specialized AI expertise can use sophisticated if-then analytics fueled by large data and information assets—to gain both deep foresight and insight to aid decision-making.

Learn more about the science behind predictive analytics in my webinar, Using Data Science to Predict the Future or in this case study: Using predictive analytics to live better lives. Find more details on insights discovery in the article, With Prescriptive Analytics, the future ain’t what it used to be. There is also a high-level analytics checklist to guide your analytics journey, which is available from EDUCBA.

Dr. Kirk Borne is a GovLoop Featured Contributor. He is the Principal Data Scientist and an Executive Advisor at management consulting firm Booz Allen Hamilton since 2015. In those roles, he focuses on applications of data science, data management, machine learning, and AI across a variety of disciplines. You can read his posts here.