The Future of Open Data and Its Relationship with the Private Sector

I’d like to discuss an open data topic that I believe does not get enough exposure. Quite frankly, it has not been researched, investigated or taken as seriously as it should. Let’s talk about the role that the private sector can and should play in ensuring open data reaches its fullest potential in the public arena.

When I was the Chief Technology Officer of a federal agency in Washington D.C., a press release was issued announcing my hire as the new Chief Analytics Officer for the City of New York. I hadn’t even started my job yet and a lot of people were reaching out to engage me. These people included city officials, NYC civic tech community members and vendors that both have and don’t have a presence in NYC.

I sat down with a private sector company in D.C. to discuss my upcoming role in NYC. The company explained to me how awesome they were and how much they could do for NYC with respects to data and analytics. During the course of this conversation, someone mentioned that there should be a tiered for-pay open data model. This model required users to pay for open data if an agency spent more time and resources to make it higher quality. When they spoke of users, they meant any company or organization that could afford it. I was incredulous and immediately shot back that setting up this kind of model was not open data and was not going to be pursued under my leadership.

I might have been correct at the time, but a few years and many, many debates later, I think there actually is a way to have a for-pay version of open data.

Before I dive into the meat of this discussion let’s set this conversation by identifying some high-level general phases of the process for opening data in a city:

• Phase 1 – Data Discovery: This is when the city agency spends time identifying what data they have that can and should be made public. This also includes identifying what should not be made public.
• Phase 2 – Extract, Transform, Load. This is the phase where the agency cleans up the data and stages it so that it is ready for release.
• Phase 3 – Presentation: Here the agencies review all the work needed to successfully release the data onto a public portal.
• Phase 4 – Access and Download. The data is finally made available, possibly in a tiered structure, for the people who need it.

One thing that the open data team in NYC and I used to discuss constantly was how we could engage the private sector to be more involved in open data at all phases in the open data process. We discussed a solution where private entities would be able to “sponsor” a data set for release to the public. In my experience in government, the biggest challenge that many open data teams face is the lack of both personnel and financial resources. If a company (or non-profit, or academic institution) decided that a data set a city managed was extremely important to them, they could then donate either time or resources to help that agency release the highest-quality version of that data set. This would allow for private sector entities to no longer just be around for phase 4 of the open data process, but also to be involved in all of the phases from the beginning.

You may be saying to yourself, “All this does is prioritize the release of data sets based on which ones are getting ‘sponsored.'” Yes, this is true, but think about it. Right now, data sets in each city are getting prioritized for release already. Some processes are transparent, but many aren’t. This process would be transparent.

If there turns out to be a high correlation between “sponsored” open data sets and download/usage counts, then you now have a successful model that can be scaled. The city can focus on identifying sponsorship that can drive productive usage. The end game of open data is more people accessing and using government data. I can see this “sponsorship” model as one of a few possible options that can get us there.

The point of this blog is to make it clear to all that in order for open data to grow and reach its fullest potential, it will have to undergo many innovative changes. As long as the main goal of democratizing access to data is still in order, I believe that we should be ready and prepared to shift how we think about open data.

Amen Ra Mashariki is part of the GovLoop Featured Contributor program, where we feature articles by government voices from all across the country (and world!). To see more Featured Contributor posts, click here.