Unlocking Budget Data in Australia: the BudgetAus Collaboration

Republished with permission from the International Budget Partnership blog

This post was written by Rosie Williams of InfoAus.

Budget transparency in Australia has recently taken a big step forward with the first ever release of federal budget data in machine readable format. Prior to this year, budget data in Australia had been locked away in PDF and Word documents. While these publications met the broad guidelines for reporting government spending to the public, analysis of government spending remained a difficult and time consuming process.

Providing information is one thing, making it usable is yet another.

Unlocking the data

As a novice programmer with a degree in sociology and background in activism, I decided to address this problem by creating a web tool that would allow users to explore the entire federal budget. The website — BudgetAus — works in much the same way as a search engine: users can search for their areas of interest to see how much money the government is spending, regardless of the agency or portfolio in which the spending occurs.

The original site was built from budget data that I manually copied and pasted from the existing PDF’s published by the government. The following year we tried to program scripts to scrape the data, but this proved too time consuming. The complexity of the data contained within the documents, and the fact that the documents presented information in different ways and were not broken down to the same level, proved challenging.

Behind the scenes, people had been working within government to release budget data in machine readable formats (as data files). However, they faced the same set of challenges – inconsistencies in the way the data was organized by different agencies made them unsuitable for use by programmers.

A budget visualization created using BudgetAus data. From Arthur Street’s Australian Budget Explorer.

Building a network

Having established my interest in budget transparency over the past year or so, I found a small network of people with a strong interest in what I was attempting. This network includes experts who work on the federal budget, veteran journalists, and professional programmers.

With the first release of machine-readable budget data imminent, we made a big push to have this data reformatted and made consistent with the requirements of BudgetAus and similar projects. This was no easy task, with a team working overnight with the Excel tables contributed by each of 180 agencies to produce line item data in a suitable format.

Going public

Getting the data is only one requirement of a successful budget transparency project. Engaging the wider public with the purpose of having access to the data is also crucial. I used a budget night event to find collaborators willing to put the budget data to use. With the help of some prominent independent journalists, Wendy Bacon and Margo Kingston, the BudgetAus collaboration, as it has become known, spent budget night using social media to find out what sort of budget questions people wanted answered.

Wendy set up a Question Bank on GitHub – an online, open source collaboration tool. This seems to be functioning quite well for public discussion of budget transparency questions. Some developers in our network set up a data visualization repository to support this and future efforts by coders and citizen bloggers to produce meaningful graphs and visualizations based on open data.

Everyone played complimentary roles, from the budget experts who providedbackground on the nitty-gritty of budget questions, to the media and our coders. Collaborators seemed to fall quite naturally into their respective functions.

Where to from here?

Based on this years’ experience of working with BudgetAus, the government is now designing a standard way for agencies to report spending.

While BudgetAus and its collaborators have helped to shine a light on the important issue of data consistency, there is much work that remains to be done. Answering questions such as how spending promises (estimates) differ from actual spending, and how different political parties make changes to public spending, will require retrospective data that is so far not available. To continue to build on the success of the project will require funding the formalization of a group working on these issues.

In the end it took leaders within government, the respective agencies, citizen journalists, citizen hackers, and the general public to begin a functioning budget transparency project. I hope that this is just a beginning.