In Defense of Data Centrism

In the never ending search to know “what works,” we have a few choices. We can look to theories, i.e., this ‘should’ work; or we can look to data. Often the latter choice is considered backward looking, or stripped of context. Data autopsies are conducted with the results analyzed and presented as a ‘case study.’ Here is what happened with BP, or Enron, or the 111th Congress.

The former choice, relying on theoretical principles, is considered by some more noble and can be expressed simply: spread democracy, protect the borders, cut taxes, etc. Theories that are believed to represent fundamental levers of reality – when pushed in a certain direction, desired outcomes result. These theories are often referred to as ‘common sense.’ Of course, yesterday’s common sense included such principles as racial inequality, ignorance to environmental stewardship, ever-rising housing prices, sexual preference as a preference, etc. It takes frightening shocks to the system to shake our faith in such levers.

Yet, ever hopeful, we press on to learn what levers control our universe. What works, and why? And how can we scale it? The answer, I believe, comes from data. But not just any data. And not through data autopsy and case studies.

Take education. We can approach improving education as a principled journey, applying common sense: reduce class size, return to single-gender classrooms, dress them up in uniforms, etc. Or we can turn to data. Yes, we can do both, but levers must be informed and confirmed by data. As a friend tells me: The data must precede the framework.

In education, however, we have a paucity of data. In a conversation with a senior official at the Department of Education last year, I discussed our shared idea of data platforms until he stopped me mid-sentence: “You’re assuming we have the right data.” No, I didn’t, but he was right. I was designing the platform before taking on the fight to tease out relevant data about student performance.

Even that phrase, student performance, is loaded with assumptions. Performance as measured by what? Standardized tests? Only last year did the majority of U.S. states agree to a common set of performance standards – and only then applicable to middle-school math and English. As to how students are assessed against these standards? That remains in debate, currently there are two clusters of states reviewing approaches to common assessment regimes. We are years from a U.S. approach to these fundamental levers for K-12 education: What is the standard against which student performance is measured, and how is that performance measured? (I should acknowledge a competing theoretical construct that opposes any national approach to education – again, I seek the data here.)

It gets worse. Institutions of higher education find an increasing number of applicants, year over year, lacking in the skills needed to succeed in their first-year studies. The resulting ‘remediation’ classes are nothing more an extension of high school. However, talk with those in the field of education, and they will tell you that K-12 schools have no common tasking from higher education regarding what is considered an acceptable skill set. While we work to get to a U.S. approach to these fundamental levers for K-12 education, this effort is not coordinated with the expectations of universities and colleges – who themselves do not agree on the answer to that basic question.

It gets worse. A recent study by IBM surveying CEOs found that the most pressing challenge is the complexity of their enterprise and industry, and the most necessary skill is and will be creativity. The ability to think critically, understand variables, and make decisions amidst uncertainty. Meanwhile, the fundamental levers we believe are necessary for K-12 education are descendants of the Trivium – the triad of grammar, logic and rhetoric developed first to shape medieval liberal arts students. A consortium of technology companies are working to develop the definition of ‘21st century skills’ they believe are necessary for their incoming labor force – but in the radically localized Education field, there is no King to accept their input. One large government contractor laments: we would like to remain an American company, but we need 70,000 engineers over the next ten years. How can we accomplish both goals, when only one represents shareholder needs? A senior education administrator meekly suggests the adoption of international standards, PISA, as a baseline for common standards only to be scolded by a peer; “But this is America!”

How do we untangle education? By fighting over which fundamental levers matter, or turning to relevant data?

There is hope.

A student returns home from a day struggling to master Algebra as her teacher struggles to increase comprehension, while not ‘teaching to the test.’ The results of this test will drive the reputation of, and government investment in, the school district. The reputation of the school district will drive housing prices, and shape neighborhoods. All are exhausted by the end of the school day, but the data collected at the point of learning remains tiny ovals filled in by a child’s number two pencil.

Returning home, the student unwinds by loading up a multi-player online immersive video game. The players navigate a complex environment, their interactions driving the direction of the game, as the game algorithms respond to player progression through the landscape. Each move is measured, assessed, and the game evolves along one of a thousand paths – this path of learning is determined by the player’s interactions, both with their computer environment and with one another. The players are connected via voice connections, as they work as a team to navigate the game’s landscape – often matched up against a set of adversaries, a mirror-image team tackling the same challenges and competing with them.

The next morning, the student loads her textbook-laden rucksack and trudges off to sit in a classroom designed during the Victorian-era, hoping to color in the ovals correctly before Christmas.

Which experience better prepared her for the ability to think critically, understand variables, and make decisions amidst uncertainty? What data matters in this story?

Let’s start here.