RSS reversed: From Feed to Blog
We all use RSS mostly to read updates from different sites. What if we could reverse this, and use RSS to actually populate a site?
Here is a story of RSS, in reverse. Maybe we should call it SSR instead of RSS…
Both for work, and in my spare time, I scan the news (and the Internet as a whole) for humanitarian, aid and development articles.
1. Using RSS the conventional way
Up till recently I exclusively used PageFlakes, a simple tool allowing you to display a multitude of RSS feeds from different websites on one page.
This gave me an instantaneous overview of what is being published in the world. But, I needed to visually scan the different feed portlets. And I had no history: once a news source is refreshed, new items come up and old ones are popped off the list. You missed it, too bad…
And it displayed “any news”, not just humanitarian news. I could not really filter out contents of interest to me, nor could I search the items.
2. Using RSS to populate a site, the script way
Thus I made The Other World News, which aggregates news from different sources, and displays it in a straight text format.
How to do this? Pretty simple:
I identified about
100 300 online news sources, covering the topics I was interested in. I took the RSS feeds of all these sites and combined them into several “streams of data”, one stream per type of news source, using a free tool called NewsGator .
For each stream of data, Newsgator lets you do some basic formatting, identifying where you want the data, time, title, text contents and lets you trim the length of the output. The actual output is a script, which looks like this:
I used that line of code in a post, or on some of my blogs I put it in a side column. Each time the page is loaded, Newsgator runs the script, and displays the latest news items.
That works well and displays only the latest humanitarian news. But, once again, it did not store the data, it was “only” a way to display the latest news items.
3. Using RSS to populate a site, the geeky way
So I needed something more: a way to keep all news posts stored, so they were search-able, tag-able, re-use-able… And here is where I am very pleased with myself. Ha! A nerd’s hand is so easily filled…
The fruits of my labour:
Aaahh. Now we are talking! Those of you following “The Road” regularly, know I have been working on these three sites – I call them my meta-sites – for a while… It took me sweat, blood and tears before I had the three sites as I wanted:
I have a site AidNews gives the humanitarian news (only). Each feed item is automatically stored as a summary, with a link to the original press article. Occasional pictures go in it too.
For Those Who Want to Know does the same, but the 250+ sources are blogs, humanitarian sites and press releases from development organisations, etc….
And while I was at it, AidBlogs aggregates summaries of the latest blogs from my fellow aidworkers.
Because I am a humanitarian nerd, I have to tell you how it works. I just gotta…
Just like in chapter 3, I used about my hundreds of online news sources (the quality of your input defines the quality of your output). I take the RSS feeds of all these sites and run them through NewsGator to combine all of them. This time, the output is a new RSS feeds rather than a script.
I take several Newsgator feeds which I combine into one feed, filter out formatting problems, delete duplicate news items, and take out only the most recent 30 posts, using Yahoo Pipes. Talking of nerd-tools, “Yahoo Pipes” is a good example! But with a great user interface:
If you ever feel geeky, and you want to play with RSS feeds, give “Pipes” a go.. You will be amazed about its powers.
Once more, Yahoo Pipes generates a new RSS feed, which I import into Tumblr, a very simple but powerful blogging platform. What is unique about Tumblr, is the built-in ability to import RSS feeds, which they convert into the posts.
Taraa…! That gives me what I want: every hour, I get a dozen new news articles in one blog. Automatically…! The big advantage is that all these posts are searchable. Here is an search on “Sudan” from my AidNews site.
Through time, my blog network grew more and more complex, check out my blog diagram for an overview of what more magical things I did with RSS feeds.
One word of caution though! Whatever you republish, taken from RSS streams from other sites, ensure you always link back to the original site and article. Otherwise, you are leeching: benefiting from the labour of others.