April, 2009

Ideas / Mash the State — 1 Comment
28
Apr 09

Hillingdon Council creates an RSS feed for every page

hillingdon-subscribe-to-page
Ever wanted to track the changes to a webpage but found there was no easy way of doing it? The Hillingdon Council website makes this easy by generating an RSS feed for every page.

At the bottom of each page there’s a “Subscribe to this page” link and feed icon. The site also makes these feeds easy to find by putting them in autodiscovery tags, providing a consistent way to subscribe in each browser.

Using RSS as a mechanism for receiving page status updates makes much more sense than writing a custom subscription system and requiring user registration such as on Sutton Council’s website.

There’s plenty of scope for Hillingdon to produce more comprehensive specific feeds for other uses but this is a very useful feature in its own right. Combined with a feed-to-email link on every page to a service like Feed My Inbox it could see a lot of usage.

Let’s also remember that there’s more to life than RSS. Other feed formats and APIs are more appropriate for different types of data such as iCalendar for events.

How-To Guides / Local Government / Mash the State / open data — 4 Comments
22
Apr 09

Top RSS tips for councils (and everyone else)

1. Validate your feeds

It only takes a moment to validate a feed. Invalid feeds can cause all kinds of unexpected weirdness in feed readers and other applications. Find any errors and fix them.

2. Use autodiscovery

People that use feeds a lot love autodiscovery. It provides a consistent way of finding and subscribing to feeds from any website. Put an autodiscovery <link> tag on your home page for every feed you’ve got and a tag on every interior page that’s got its own feed, eg. a tag for the news feed on the news page.

The standard tag format for RSS autodiscovery is:

<link rel="alternate" type="application/rss+xml" href="http://www.anytown.gov.uk/news.rss" title="Anytown Council News" />

and for Atom autodiscovery, use:

<link rel="alternate" type="application/atom+xml" href="http://www.anytown.gov.uk/news.atom" title="Anytown Council News" />

3. Use standard feed icons but only as direct subscription links

Whether you’re using RSS or Atom feeds, the standard feed icons are the common “beacon” style ones from Feed Icons. Get rid of the old orange rectangular “RSS” and “XML” text icons if you have them — they’re obsolete.

But don’t use the feed icons as illustrations. They should be clickable links directly to the feeds themselves so that people can subscribe. Avoid using them to link to RSS help pages or anything else.

Think: When I see this icon, I can click on it to subscribe.

4. Put your feed icons at the top of the related content

In-page feed subscription icons should be placed as near to the top of the related content as possible. Don’t bury the icon at the bottom of a list of news headlines, or even worse, in your page footer or a sidebar. Try to reinforce in your readers’ minds that the feed is an alternative way of viewing the same content.

5. Check your feeds in an RSS reader

This will often show up odd things in your feed that a validator won’t catch — like all your item dates being the same, for example.

6. Don’t lose your subscribers when your feed moves

If your CMS will let you, publish your feeds at permanent URLs. If it won’t, you’ll lose all your subscribers when you move to a new CMS and the URLs change.

If this happens, use an HTTP 301 redirect to tell your readers’ clients that the feed URL has changed permanently. Here’s how you do it.

Alternatively, proxy your feed through FeedBurner which will give it a permanent URL (and a few other toys to play with like usage stats, too).

7. Don’t offer the same feed in more than one format

People have got better things to do than to try to decide whether they want to read a feed in RSS 2.0, RSS 1.1 or Atom 1.0 format. Choose a single format for your whole site and stick with it. In practice, all formats work in all applications anyway. It gives you less to maintain and one less thing to worry about.

8. Shorten your item links if they’re longer than 255 characters

Some RSS reader applications use 255-character long fields to store item links. Some links are longer than that and will be truncated (and therefore, broken). If your CMS serves up item URLs longer than 255 characters try to run them through a URL shortening API like TinyURL first so they’ll always work.

9. Write a sensible feed title

The bit in your feed’s <title> element is what gets displayed as your feed’s title in RSS readers and other applications. Usually this should be your organisation’s name and a short indication of the content, eg:

  • Anytown Council News
  • Anytown Council – what’s on
  • Anytown Council Job Vacancies

Far, far too many feeds just have either the “Anytown Council” or the “news” bit, forcing users to rename those subscriptions in their readers if they’re going to make any sense at all.

Here are some bad examples from the Mash the State database:

  • Council Website Updates (but which council?)
  • Events (for who?)
  • Jobs Vacancies (sic)
  • Latest Online Consultations
  • Travel Information
  • www.anytown.gov.uk::Latest News (get rid of all that clutter)

10. Create an easy subscribe-by-email service

Many councils provide email alert services to keep their residents up to date. This is a lot of hassle, having to deal with bounces, unsubscribes and maintaining the whole mechanism. If you’ve got RSS feeds it only takes a moment to let people subscribe to them by email using third-party services like FeedMyInbox.

You can create a direct subscription link like this:

http://www.feedmyinbox.com/?feed=http://www.anytown.gov.uk/news.rss

This is how we do the black subscribe-by-email links on Mash the State’s council pages. You get a whole email service for nothing without having to pay a penny or do any more work at all. Just like with RSS in general.

Got any more good tips? Leave a comment and I’ll work them into the article here.

Enjoy.

Uncategorized — Comments Off
20
Apr 09

Example “House Rules” for community forums

These are the (slightly modified) house rules I developed for Sutton Chat. If you’re starting a new community forum or blog and would like to use them as the basis for your own rules, please take them and modify them to suit you while giving attribution to Adrian Short under the Creative Commons Attribution 2.0 UK licence.

—————————————

House Rules for Anytown Chat

My aim is for Anytown Chat to be a place where everyone can feel comfortable debating both serious issues and the lighter side of life. In order for this to happen, there are a few House Rules which will be enforced sensibly.

By registering with this website you agree to follow these rules.

1. Be yourself

(Snip this clause if you’re happy to have pseudonymous members.)

Use your real name as your username when registering with Anytown Chat. Anytown is a real place full of real people, many of whom know each other in real life. Using real names rather than nicknames allows members to identify others that they already know and get to know people that they don’t. It also ensures that members are accountable for their words: If you wouldn’t put your own name to a comment, it probably doesn’t belong here anyway.

You are encouraged but not required to show your face by creating a profile picture of yourself.

2. No bad language

Most of us have a broad vocabulary of swear words but Anytown Chat is not the place to use them. Avoiding bad language helps to keep a civilised and intelligent tone to discussions. Use your imagination and where that fails, just restrain yourself.

3. No personal attacks

Anytown Chat is about sharing information, learning and debating. It’s not a place for personal disputes and vendettas. By all means strongly dispute others’ ideas and arguments but if you make it personal you’ll be asked to stop.

4. Respect others’ privacy

This is a public website and everything you write here can be viewed by anyone. Practically, things written here will be permanently available to the rest of the world. Do not disclose any private or personal information about other people, whether they are members here or not. This isn’t Facebook or your private email. The whole world can see what you’re writing.

5. Avoid discrimination

If you hold any unpleasant bigotries about people on the grounds of their sex, sexuality, age, nationality, ethnicity or (lack of) religion, this isn’t the place to express them. Get yourself a blog if you really must. These topics will inevitably come up in discussions but I hope that everyone is able to debate them without making the site uncomfortable for others to participate.

6. No porn

This is a site for adults, not an “adult site.” Don’t post porn, whether words, pictures, videos or links to any of these things.

7. Respect the law

Hate speech, libel, incitement, copyright infringement and obscenity are all forbidden here.

8. No spam

Don’t post just to advertise your website or business. If in doubt, please ask first [create a link here to your email address or contact page]. It’s fine to use your business or professional web address in your member profile.

These rules will be reviewed and changed if necessary in the light of experience.

Enjoy yourself

While it’s not a rule, I hope you enjoy chatting here and that these rules enhance rather than inhibit that enjoyment.

How-To Guides / Local Government / Mash the State — 5 Comments
20
Apr 09

Comments not allowed at your council website? Here’s how to answer back

UPDATE 27 Feb 2010: The Boris Backchat blog mentioned in this post has served its demonstration purpose and has now been deleted.

A few people have raised the objection that what Mash the State is currently doing with council RSS feeds is really just helping councils to deliver their PR (or as those critics often like put it, “propaganda”).

In one sense, they’re right. A council’s press releases or “news” are just their own side of the story. You’d have to be pretty naive to think otherwise.

But getting any kind of information out into the open where it can be scrutinised, compared, cross-referenced and easily discussed is always an advantage. Here’s how to build a discussion blog around your local council’s news. Of course, if they don’t have an RSS feed this isn’t possible, which is why Mash the State exists in the first place.

Time required: Around 15 minutes.

Skills required: Just basic web use stuff. No programming or HTML. Anyone online should be able to do this.

Here’s one I made earlier: Boris Backchat. Got something to say to the London mayor? Just leave a comment.

Apologies to those outside London — I had to choose something!

Here’s how I did it:

1. I registered a new blog on WordPress.com. This is free and only took a moment.

2. I found the URL (web address) of Boris’s RSS feed. Visit your local council or other government website and hover your mouse over the RSS feed link or icon. Right-click and choose “Copy shortcut” (Internet Explorer) or “Copy link location” (Firefox) or whatever your browser gives you in the right-click menu.

In this particular case it was easier to grab the feed URL from the Greater London Authority Mash the State page.

gla-screenshot

3. I signed up at xFruits which has a whole set of free tools to do things with RSS feeds. This is free.

4. I used the “RSS to my blog” tool on xFruits which automatically copies the contents of an RSS feed into a blog, making a new blog post for each item in the feed.

xfruits-rss-to-blog

First I typed a title for the new blog site and a few tags.

xfruits-step-1

Then I pasted in the feed URL that I’d copied in step 2.

xfruits-step-2

To configure this I also needed the URL of my new blog’s “API endpoint”. This is the address which other programs can use to push data into your blog.

The API endpoint URL for this blog is:

http://borisbackchat.wordpress.com/xmlrpc.php

The format is the same for all blogs on WordPress.com:

http://yourblogname.wordpress.com/xmlrpc.php

I also had to type in my WordPress username and password, and as I’ve got several blogs on WordPress, had to choose the right one to send the RSS feed to from the drop-down menu.

xfruits-step-3

5. And that’s pretty much it. I went into the WordPress settings and set the time zone correctly and edited the site description. Now we’re ready to go.

To close the loop, if you want to keep up with the latest posts on Boris Backchat you can subscribe to both the new posts (articles) and comments in your RSS reader.

xFruits will work with most popular blog systems including Blogger, TypePad, Movable Type and WordPress hosted on your own server.

Welcome to open government. :)

Thanks to Jon Bounds on Twitter for tipping me off about xFruits. It’s a great set of tools. Jon has just set up a similar site for Birmingham City Council.

… and as I always like to say about these things, it’s taken longer to write about it than to do it!

Citizenship / Web design — Comments Off
14
Apr 09

Why I’m throwing down the gauntlet to our councils over RSS feeds

mtslogo_200

You’re free to republish this article under the Creative Commons Attribution 2.0 UK licence with credit and a link to Adrian Short / Mash the State

Today I connected 66 councils to their citizens by making it easy to subscribe to their news by email. It took me around ten minutes. I’d say this was a fairly good use of my time in terms of the ratio of effort to value produced, but I can’t claim to have done it single handed. What made it possible is that all 66 of these councils serve an RSS feed from their websites — and they’re the only ones in the country that do. Hooking those feeds up to FeedMyInbox through the council pages at Mash the State was a simple matter of dropping a single web link into a template and pushing it to the live site. Job done.

RSS is a simple way of getting data out of a website and into another program. The technology is ten years old and RSS feeds are ubiquitous on blogs, on mainstream news media websites and in Web 2.0 applications. The three leading web browsers — Internet Explorer, Firefox and Safari — all contain built-in RSS readers. Yet despite running websites costing tens of thousands of pounds annually each, only 15% of UK councils bother with RSS. Nothing could be more symbolic of large parts of government’s unwillingness to think beyond the confines of their own websites than making it practically impossible to receive basic local council information like news and events except by taking a trip to anytown.gov.uk to do it on the council’s own terms.

The ten minutes it took to emailify those 66 councils compare quite unfavourably with probably a similar number of hours I’ve spent trying to scrape Sutton Council’s news into a database, and from there through Delicious into RSS and Twitter. Writing screen scrapers — programs which extract text from web pages and turn them into structured, reusable data — is sometimes tricky but Sutton’s news is trickier than most. The news archive serves inconsistent page structures and even dynamically changing URLs to compete with. I vowed never to write another scraper, though as we’ll see, that’s a promise I soon had to break.

Screen scraping and copyright infringement are the dirty not-so-secrets of the civic hacking world. Show me a useful, innovative third-party civic website and I’ll most probably be able to show you the terms and conditions that were ignored and the data that was taken and repurposed without permission or legal licence. Similar behaviour is not unknown in the public sector itself, in some cases because government organisations are recycling that very same stolen data from third party applications into their own websites. The recent Rewired State National Hack the Government Day saw some incredibly inspiring, innovative and useful projects produced in very short order. How many of these projects didn’t involve citizens jailbreaking their own government to get the data they’ve paid for? What kind of society not only massively impedes but actually criminalises — in principle if not in practice — citizens devoting their own time, skills and money to write software to improve democracy and public services? Our society, it seems.

This has to stop. Hackers have shown their ability and willingness to surmount technical obstacles and run legal risks to get the data they need but less technical citizens simply cannot. No-one should have to. A rich, technologically-advanced and supposedly forward-thinking society such as ours should make citizens’ access to government data so commonplace that it doesn’t deserve comment. No technical wizardry required. No legal minefields to navigate. Just all the data served through common protocols with open licences that permit, well, anything. Then we can focus our time and energy on the considerably more interesting higher-order opportunities that come from actually using government data, not just getting hold of it.

Last week I launched Mash the State, a national campaign to get government data to the people. It’s not a new idea but our method is. We’ll be setting up a series of challenges to the public sector, asking one group of public bodies at a time to release one specific set of data. Our first challenge asks all local councils to serve up an RSS news feed by Christmas. I wouldn’t have bet good money in 2003 that by 2009 370 councils would still be without RSS, but here we are. I’ve thrown the gauntlet down and I’m pleased to see that a couple of hundred people have signed up to our website or followed us on Twitter to help make this happen. The councils have got over eight months to do what in most cases will not be more than half a day’s work to serve RSS from their websites. Others less fortunate will have to persuade their content management system suppliers to enable this feature for them. All have got plenty of time to perform this technically trivial task in time to give the public a small but highly symbolic Christmas present that shows that government in this country is prepared to trust its citizens with their own data.

As for my promise never to write another scraper, it didn’t last long. The very first task to build Mash the State was an hour spent writing a scraper to tease a list of councils from a government website. Join us and help to hasten the day when no-one will ever have to do anything like that again.

Local Government / Mash the State / open data / Strategy — 8 Comments
14
Apr 09

Why I’m throwing down the gauntlet to our councils over RSS feeds

You’re free to republish this article under the Creative Commons Attribution 2.0 UK licence with credit and a link to Adrian Short / Mash the State

Today I connected 66 councils to their citizens by making it easy to subscribe to their news by email. It took me around ten minutes. I’d say this was a fairly good use of my time in terms of the ratio of effort to value produced, but I can’t claim to have done it single handed. What made it possible is that all 66 of these councils serve an RSS feed from their websites — and they’re the only ones in the country that do. Hooking those feeds up to FeedMyInbox through the council pages at Mash the State was a simple matter of dropping a single web link into a template and pushing it to the live site. Job done.

RSS is a simple way of getting data out of a website and into another program. The technology is ten years old and RSS feeds are ubiquitous on blogs, on mainstream news media websites and in Web 2.0 applications. The three leading web browsers — Internet Explorer, Firefox and Safari — all contain built-in RSS readers. Yet despite running websites costing tens of thousands of pounds annually each, only 15% of UK councils bother with RSS. Nothing could be more symbolic of large parts of government’s unwillingness to think beyond the confines of their own websites than making it practically impossible to receive basic local council information like news and events except by taking a trip to anytown.gov.uk to do it on the council’s own terms.

The ten minutes it took to emailify those 66 councils compare quite unfavourably with probably a similar number of hours I’ve spent trying to scrape Sutton Council’s news into a database, and from there through Delicious into RSS and Twitter. Writing screen scrapers — programs which extract text from web pages and turn them into structured, reusable data — is sometimes tricky but Sutton’s news is trickier than most. The news archive serves inconsistent page structures and even dynamically changing URLs to compete with. I vowed never to write another scraper, though as we’ll see, that’s a promise I soon had to break.

Screen scraping and copyright infringement are the dirty not-so-secrets of the civic hacking world. Show me a useful, innovative third-party civic website and I’ll most probably be able to show you the terms and conditions that were ignored and the data that was taken and repurposed without permission or legal licence. Similar behaviour is not unknown in the public sector itself, in some cases because government organisations are recycling that very same stolen data from third party applications into their own websites. The recent Rewired State National Hack the Government Day saw some incredibly inspiring, innovative and useful projects produced in very short order. How many of these projects didn’t involve citizens jailbreaking their own government to get the data they’ve paid for? What kind of society not only massively impedes but actually criminalises — in principle if not in practice — citizens devoting their own time, skills and money to write software to improve democracy and public services? Our society, it seems.

This has to stop. Hackers have shown their ability and willingness to surmount technical obstacles and run legal risks to get the data they need but less technical citizens simply cannot. No-one should have to. A rich, technologically-advanced and supposedly forward-thinking society such as ours should make citizens’ access to government data so commonplace that it doesn’t deserve comment. No technical wizardry required. No legal minefields to navigate. Just all the data served through common protocols with open licences that permit, well, anything. Then we can focus our time and energy on the considerably more interesting higher-order opportunities that come from actually using government data, not just getting hold of it.

Last week I launched Mash the State, a national campaign to get government data to the people. It’s not a new idea but our method is. We’ll be setting up a series of challenges to the public sector, asking one group of public bodies at a time to release one specific set of data. Our first challenge asks all local councils to serve up an RSS news feed by Christmas. I wouldn’t have bet good money in 2003 that by 2009 370 councils would still be without RSS, but here we are. I’ve thrown the gauntlet down and I’m pleased to see that a couple of hundred people have signed up to our website or followed us on Twitter to help make this happen. The councils have got over eight months to do what in most cases will not be more than half a day’s work to serve RSS from their websites. Others less fortunate will have to persuade their content management system suppliers to enable this feature for them. All have got plenty of time to perform this technically trivial task in time to give the public a small but highly symbolic Christmas present that shows that government in this country is prepared to trust its citizens with their own data.

As for my promise never to write another scraper, it didn’t last long. The very first task to build Mash the State was an hour spent writing a scraper to tease a list of councils from a government website. Join us and help to hasten the day when no-one will ever have to do anything like that again.

Local Government / Mash the State / open data — 5 Comments
8
Apr 09

Mash the State is live!

Good evening everyone,

It’s nearly bedtime so this is just a brief hello and welcome to all the early-adopting state mashers out there.

If you’re wondering what this is all about, I hope most of your questions are answered in the FAQ. If not, please ask them here and I’ll do my best to answer them.

For now, this is also a good place to leave any general comments about the campaign and the website.

Thanks for coming along and let’s hope we can build this into a successful, sustainable campaign. Right now, it’s going to be one council and one RSS feed at a time. Only 370 to do before Christmas!

← Older Entries