We know that developers will be looking to reuse open data for the OS Open Data and OS Open Space Challenge (not the only challenge, see here for the other one!) for ourcompetition event. So we asked @vacantimouse from ScraperWiki to describe what a tool that enables people to reuse and refine open data. Many thanks to Nicola for this guest blog!
We don’t just help you get data, we help you to build something with it
At ScraperWiki we don’t just help you get data, we help you to build something with it. If you know Ruby, Python or PHP, we have made for you your own developer environment for scraping data. It’s all in-browser and it’s free to sign up. If you’re new to scraping, just check out some of our tutorials and documentation.
As part of DotGovLabs, your mission is to design or develop prototypes for ideas to improve your neighbourhood. A good example of this type of data work in action comes from our Glasgow Hacks and Hackers Hack Day. The Guardian local beat blogger for Edinburgh,Michael MacLeod, paired with a programmer, Robert McWilliam to scrape all the Edinburgh planning applications. They put it all on a map for everyone to easily access what planning is going on in their area!
So what does all this ‘scraping’ mean?
A scraper is a type of programming script, something written in code, which fetches data from a website. With ScraperWiki you can put this data into a datastore, structured any which way you write your programme. You can schedule your scraper to go fetch the data at regular intervals i.e. when you expect more data to be released.
That way when you build an application using the datastore from your scraper, the application stays up to date. Once you have the data coming in you can even put it out on Twitter for anyone to follow the data. This was done for food safety inspections in Walsall. Check out an easy way to make a twitter account for public data but reading the blog post.
If you want to be even more adventurous, you can make the data speak! No matter what you want to do, if you need to get at data on the web head to ScraperWiki.com.
Over to you!
Do you want to design or develop a prototype on how people can help each other. Check out our competition and here and sign up to our event!
Leave a Reply
You must be logged in to post a comment.