Suggestions for governments stepping into open data

I’ve been completing a survey for the Spatial Industries Business Association (SIBA) related to the Queensland Government’s open data initiative, where one of the questions asked Can you list or describe any learnings that would be useful in Queensland?

I’ve provided a number of my thoughts on this topic, having closely observed open data initiatives by government over the last five years, and written periodically on the topic myself, such as:

• Mapping open data site generations
• Is there really an open data El Dorado?
• How should governments educate agencies about open data?
• Infographics: How does Australia compare on government open data released?
• From open data to useful data
• Is it easy for non-programmers to reuse government open data?

To share the thoughts I placed in the survey more broadly – for any value they have for other jurisdictions – I’ve included them below:

• Data released in unusable formats is less useful – it is important to mandate standards within government to define what is open data and how it should be released and educate broadly within agencies that collect and release data.
• Need to transform end-to-end data process. Often data is unusable due to poor collection or collation methods or due to contractual terms which limit use. To ensure data can be released in an open format, the entire process may require reinvention.
• Open data is a tool, not a solution and is only a starting point. Much data remains difficult to use, even when open, as communities and organisations don’t have the skills to extract value from it. There needs to be an ongoing focus on demonstrating and facilitating how value can be derived from data, involving hack events, case studies and the integration of easy-to-use analysis tools into the data store to broaden the user pool and the economic and social value. Some consideration should be given to integrating the use and analysis of open data into school work within curriculum frameworks.
• Data needs to be publicly organised in ways which make sense to its users, rather than to the government agencies releasing it. There is a tendency for governments to organise data like they organise their websites – into a hierarchy that reflects their organisational structures, rather than how users interact with government. Note that the ‘behind the scenes’ hierarchy can still reflect organisational bias, but the public hierarchy should work for the users over the contributors.
• Provide methods for the community to improve and supplement the open data, not simply request it. There are many ways in which communities can add value to government data, through independent data sets and correcting erroneous information. This needs to be supported in a managed way.
• Integrate local with state based data – aka include council and independent data into the data store, don’t keep it state only. There’s a lot of value in integrating datasets, however this can be difficult for non-programmers when last datasets are stored in different formats in different systems.
• Mandate data champions in every agency, or via a centre of expertise, who are responsible for educating and supporting agency senior and line management to adapt their end-to-end data processes to favour and support open release.
• Coordinate data efforts across jurisdictions (starting with states and working upwards), using the approach as a way to standardise on methods of data collection, analysis and reporting so that it becomes possible to compare open data apples with apples. Many data sets are far more valuable across jurisdictions and comparisons help both agencies and the public understand which approaches are working better and why – helping improve policy over time.
• Legislate to prevent politicians or agencies withholding or delaying data releases due to fear of embarrassment. It is better to be embarrassed and improve outcomes than for it to come out later that government withheld data to protect itself while harming citizen interests – this does long-term damage to the reputation of governments and politicians.
• Involve industry and the community from the beginning of the open data journey. This involves educating them on open data, what it is and the value it can create, as well as in an ongoing oversight role so they share ownership of the process and are more inclined to actively use data.
• Maintain an active schedule of data release and activities. Open data sites can become graveyards of old data and declining use without constant injections of content to prompt re-engagement. Different data is valuable to different groups, so having a release schedule (publicly published if possible) provides opportunities to re-engage groups as data valuable to them is released.