Posts Tagged: open data


31
Jan 11

Open data for all

There are five types of potential users for open data and data-driven apps:

  1. data experts and computer scientists who can use semantic web technologies;
  2. software developers who can use XML, JSON, etc.;
  3. power users who can use CSV, spreadsheets, RSS, KML/Google Earth, perhaps Yahoo Pipes;
  4. general users who can use a web browser;
  5. offliners who need printed materials, ambient displays, public screens etc.

Most of the focus seems to be on providing data for data experts and developers so they can build apps for general users and power users. We need more data suitable for power users to use directly and more apps for offliners. We’re all offline sometimes.

My own app for offliners is QR Code Posters which will print a poster from any RSS feed. See how it can be used here.

Sutton Open Maps caters for general users, power users and developers by showing draggable Google Maps of local features along with KML (Google Earth), XML and JSON downloads on the same page. Whether you want to just find a local recycling centre, download the data into Google Earth for a school project or build your own app from the data, you’re covered. (It’s open source, too.)

This post started life as a comment.


7
Jan 11

TfL’s information doesn’t want to be free

I’m a big fan of London’s Barclays Cycle Hire scheme. I praised it when it was introduced, I created a free API service for developers to help them get live data about bike availability to make useful apps for people, I built a realtime 3D visualisation of bike availability and I even wrote a simulator to help me better understand bike movement patterns. I still think it’s a great system and I’m keen to do what I can to help people use it and to make it work better.

So when Boris announced that the scheme had just passed its one millionth journey milestone it seemed like a good time to ask Transport for London for the journey data. It’s an easy enough job: Just a single database query to fetch the times, origin and destination of each trip. If I could load this data into my simulator I might be able to see where extra bikes and docking stations might be needed. I put in a Freedom of Information Act request, confident that I’d have the data within the 20 working days limit required by law.

That was three months ago on 8 October. I’m still waiting.

The good news is that the data has just been made available in TfL’s developers’ area and some people are already starting to do interesting and useful things with it. But behind that happy fact is another example of a public body deciding to completely ignore their Freedom of Information Act responsibilities and the rights of an applicant in pursuit of its own perceived interests.

Data delayed is data denied

Under the law, public bodies have got 20 working days to reply either with the information requested or to claim an exemption. The time limit is there for a good and obvious reason: Without it, public bodies can string an applicant along indefinitely, and with many requests being time-sensitive this can often past the point where the information would be useful.

Fortunately I didn’t have a specific deadline for using this data but it certainly would have been more useful to me sooner rather than later. I could have been working on it for two months by now. And if TfL had been keen for other developers to use it, they could have had it too. Some developers were keen to get hold of it for the Open Data Hackday on 4 December last year but that came and went without any sign of the data.

So why was the data delayed? I estimate that there would have been less than two hours work to produce it and send it to me, or to put it on an open website where anyone could download the file.

“Your free information is in this locked box. Sign this contract and if we like what you’re doing you can have it.”

The answer lies in TfL’s desire to wrap the data in a complicated contract rather than make it available to me or anyone else directly and legally unencumbered. This might make sense in the context of some data and some data users but it’s directly inimical to the aims and indeed the law of freedom of information. The data in TfL’s developers’ area isn’t open data and it’s not available to everyone. As the site says:

Please complete the registration form below to use our syndication feeds. Before we give permission to use any feeds, we need to know how they will be used, where they will be used and how many people are likely to view them.

So why should anyone have to apply for permission to get access to their freedom of information answer? Why not just send it to the applicant?

The Information Commissioner, who regulates public bodies’ compliance with the Freedom of Information Act is quite clear that information must be supplied regardless of the identity and motives of the applicant. His guidance (PDF) states:

A request therefore has to be considered on the basis that it could have been made by any person; the identity of that person is not a material consideration when deciding whether or not to release information. It is for this reason that we do recommend as good practice that requests under obvious pseudonyms should normally be considered unless there is reason to think that any of the matters below need to be taken into account.

There follows some general exceptions regarding vexatious requests, people requesting their own personal information and costs issues, none of which apply in this case.

On the issue of the applicant’s motives:

There is also no specific reference in the FOIA to the principle that requests for information must be considered without reference to the motives of the requester.

However, there are no references in the Act indicating that anyone can be asked to provide a reason for requesting information and it is from this absence that the principle [of disregarding the applicant's motives] is drawn.

The Information Commissioner then quotes the Lord Chancellor’s code of practice on freedom of information:

Authorities should be aware that the aim of providing assistance is to clarify the nature of the information sought, not to determine the aims or motivation of the applicant. Care should be taken not to give the applicant the impression that he or she is obliged to disclose the nature of his or her interest as a precondition to exercising the rights of access, or that he or she will be treated differently if he or she does (or does not).

But if I want to get a response to my FOI request from TfL I am asked to enter into a contract with them whose terms include:

2.1.2 [You shall] only use the Transport Data in accordance with these Terms and Conditions and the Syndication Developer Guidelines, and not use such information in any way that causes detriment to TfL or brings TfL into disrepute. The rights granted to You under these Terms and Conditions are limited to accessing and displaying or otherwise making available the Transport Data for the purposes stated by You in Your registration.

So not only is TfL’s contract explicitly asking me to state my motive as a precondition of access, it also constrains me from using the information for any other purpose and arguably prevents me from using that information to criticise TfL, thereby causing it “detriment” or bringing it into “disrepute”. If I don’t agree to this they can deny access altogether and if I subsequently break the agreement in their view they can revoke access. This is a funny kind of free information.

The Freedom of Information Act is designed to enable scrutiny of government. It’s inevitable that some information requested may cause embarrassment to the public body providing it or even bring it into disrepute. If the law is going to be workable at all, public bodies must consider each application on its merits alone without concerning themselves with the applicant or their motives. To do otherwise would allow public bodies to effectively pick and choose which requests they answered. TfL’s decision to require me to enter into an extremely restrictive contract with them to get a response to my freedom of information request is applicant and motive discrimination by the back door. It’s not something that should be tolerated from TfL much less adopted by other public bodies as a way to weaken FOI applicants’ rights. Free information should not come wrapped in a restrictive contract wall. That’s why I won’t be accepting TfL’s terms and I’ll simply have to leave the analysis of this Cycle Hire data in the very capable hands of others.


21
Aug 10

Armchair Auditor interview with Henry Kelly on BBC Radio Berkshire

Following on from my appearance on BBC Radio 4 PM I appeared live on BBC Radio Berkshire on 16 August talking to Henry Kelly about Armchair Auditor, my public spending software and website.

At present the site has the Royal Borough of Windsor and Maidenhead’s spending data so it was of particular interest to this local radio station.

Adrian Short interviewed by Henry Kelly


8
Jul 10

Some more facts about SpotlightOnSpend for FullFact.org

Full Fact is a website that aims to find the true facts behind the spin and obfuscation of public debate. According to them:

Full Fact is an independent fact-checking organisation. We remove the spin from political statements and make it easier to see the facts and context behind the claims made by the key players in British political debate.

Our main work is to:

  • Analyse, challenge and expose misleading claims
  • Enable people to verify or rebut claims and campaign for improved standards

So I was interested to see how they’d tackle the controversy over Chris Taggart’s criticism of the way in which Spikes Cavell are publishing council spending data on their SpotlightOnSpend website.

Continue reading →


5
Oct 09

Ernest Marples: An elegy

Ernest Marples is dead and I am pissed off.

I refer, of course, not to the erstwhile postmaster general and transport minister who retired to the grave in 1978 but to the eponymous website which has been crushed beneath the Royal Mail’s clunking fist.

Ernest, you did one thing and you did it brilliantly. You gave programmers a gizmo that converted postcodes into geographical locations. Such an unglamorous task formed the backbone of websites that provided public benefit and private delight in equal measure.

By powering Planning Alerts you let thousands get news of local planning applications where they wanted it – in their inboxes – rather than having to rummage around in the darkest reaches of their councils’ websites.

In a time of high unemployment and higher uncertainty, you gave people a fast and easy way to find vacancies near them through Jobcentre Pro Plus without having to suffer the frustrations and indignities of the Jobcentre “adviser” and the official government website.

And as the cornerstone of The Straight Choice you helped us to create a public library of election leaflets that let us judge for ourselves whether our politicians’ promises were worth the paper they were written on.

You inspired dozens of developers to create civic projects that without you would have been unthinkable and now without you may well be impossible.

At a time when joined-up government was either a breathless aspiration or an oxymoronic joke, you helped to bond parts of government that no council, ministry or quango could reach.

Above all, Ernest, you provided a glimpse of what we the people could do with free access to the data that we had paid to create and are now expected to pay for again to use.

For all this you asked for neither recognition nor recompense but just the chance to carry on doing what you loved. A chance which you were so ruthlessly and shamelessly denied.

Farewell, Ernest. You were one of us and now without you we are less.