Really Silly Syndicators

Those of us who subscribe to the brand-new Slate RSS feed got an update this evening: a new story had been added to the website. Now Slate doesn't actually put the content of its stories into its RSS feed, but does give us the headline and the standfirst. The headline, this evening, was "Vox Populi", and the standfirst was "Lying down the rabbit hole." In other words, the RSS feed was no use at all. If you wanted to have the vaguest notion what on earth the story was about, you had to click through from your RSS reader to the website.

Slate celebrated the launch of its feed with a long article extolling the virtues of RSS, but it's clear that the powers that be at the magazine still don't get it.

In fact, for all the wars between different flavours of RSS (I have no idea what the substantive difference is between 0.9x, 1.0, 2.0, and Atom, and I have no desire to find out), the biggest problem with it is not the technology itself, but rather the way that most websites use it.

Every so often, an RSS feed is actually better than the website: NewYorkish is a prime example. It breaks in my browser, with the middle column overlapping the main text, but it looks wonderful in my RSS reader. Much more frequently, however, it's the other way around.

For starters, most bloggers who don't feel comfortable messing around in the engine room, as it were, simply go with the default settings in Movable Type or TypePad. For reasons I've never been able to work out, the default setting normally gives you little more than the headline and the first 20 words, rather than the whole piece. Back in June, Lance Knobel guilt-tripped me into making sure everything I wrote made it into the RSS feed, quoting with approval a fellow blogger saying that "I get annoyed with sites that don't provide a full RSS feed and insist on offering snippets or headlines only."

But since then, the situation has only got worse. Gawker Media, for instance, has horrible RSS feeds, which never include images, links to other sites, or more than a tiny bit of text. ArtsJournal is even worse, providing nothing but a headline. And although the New York Times RSS feeds are excellent, good luck finding them from the nytimes.com homepage: they're all hosted somewhere else entirely.

In general, going down my list of feeds, the bad is much more common than the good. I limit myself to 39 feeds at any one time: beyond that I can't see them all at once in my aggregator, and in any case beyond that I'd never have time to get any work done at all. Of the 39, just 18 – Low Culture, MemeFirst, Davos Newbies, felixsalmon.com, Charles Stewart, The Trademark Blog, greg.org, Gothamist (a recent development: thank you!), Below 14th, Lockhart Steele, Bookslut, TMFTML, Anil Dash and his Daily Links, BuzzMachine, NewYorkish, Best Week Ever, and Belle de Jour – have full RSS feeds which duplicate the content on the website. And TMFTML's links often don't work from the RSS reader, for some reason.

I'm not going to name-and-shame the bloggers who don't serve nice pretty RSS feeds (Choire), because there are so many who don't serve any kind of RSS at all. The ones on blogspot we can excuse, but the likes of Andrew Sullivan, Josh Marshall and Daniel Radosh – not to mention Drudge – really should get with the program.

Then there are the corporate blogs – the ones which are meant to drive traffic to the sites on which they're hosted. Some, like The Kicker, serve crappy RSS (although to be fair, The Kicker's RSS feed is no crappier than that of Elizabeth Spiers personally), but most have no RSS at all. The Corner, Best of the Web Today, Altercation, Etc – none of these blogs seems to have realised that an RSS feed would increase, not decrease, their total traffic. In this category, too, you should include Romenesko, which, one would think, would be pretty much ideal for subscribing to.

The main reason not to put all your content into your RSS feed, of course, is if you're keen that your readers view the advertisements on your website. It's a bit like the old debate about newspapers putting their content online: they were worried that the website would cannibalise their paper sales, and now they're worried that the RSS feed will cannibalise their website. I'm not convinced, but I do see the argument, and therefore I don't have a problem with news organisations like the New York Times and the BBC which provide good, regularly-updated RSS feeds which give you the top headlines and a one-line summary of the story.

When that kind of attitude spreads to webzines like Slate, however, I get worried. Slate's RSS feed not only doesn't provide the full content of the stories: it doesn't even provide the same amount of information that's in the site's table of contents. There's a story with the headline "Vox Populi"? Um, great. Now tell me who wrote it, and I might be interested.

Slate is all about its franchises: Chatterbox, Ballot Box, Webhead, Dear Prudence. If something new is up on the site by Lithwick or Kinsley or Shafer, I'll want to read it. But the RSS feed, unlike the table of contents, tells me neither the department in which the story is located, nor the name of the author. All I'm left with is a headline, which means almost nothing. Why build up franchises only to ignore their very existence when you move to an RSS feed? Even the main image on the Slate homepage ensures that every story it mentions also comes with the name of the author.

RSS, I believe, has been around since 1999, yet remains something known only to a tiny minority of internet users. So long as the biggest and most important websites continue to treat their RSS subscribers with such disdain, this state of affairs is likely to continue. Really, I see little point in trying to fix whatever minor problems there may or may not be with RSS itself. Unless and until content providers actually bother to use what it's capable of right now, it's going to remain something used only by the blogosphere's nerdier types.

Posted by Felix at 22:06 EST

Comments

I agree with your frustration...I've been subscribing to the craigslist RSS feed ever since it popped up on the site and it hardly ever saves me from surfing the site as before, giving only the headline - even though most craiglist posts are small enough to make the "20 words" threshold, even, meaningful. what's the use?

a better solution to the problem is to do an end-run around RSS altogether. my company's new application, JYTE, might give you a lot of what you're looking for, especially for sites with "corporate" content. it's in beta right now but getting more powerful every day as we add features to the client, and scrapers and feeds to our database. instead of using the feed as a metaphor, and organizational rationale, it uses searches (keywords or source-based) to organize your headlines. Articles can be opened and read in the client. You can learn more at my website. and a note...the RSS reader was added last night.

Posted by: sarah gilbert at 2:33 EST, March 07, 2004

There are legitimate technical reasons not to place the full content of an article into an RSS feed. For a start you should restrict entries to plain text, rather than HTML, which means no hyperlinks or text formatting.

If you do place HTML in your feed you have no certainty that it will be rendered as you intended, and it presents a major security hole for any RSS client that does render it.

The best solution is to place a good descriptive title and story summary in the feed, but how easy that is varies with the software used.

Posted by: Colin Stewart at 14:31 EST, March 07, 2004

As always, an excellent analysis. The headline-only feeds are truly useless. One other possibly relevant issue: I don't think there's an (easy) way to measure the number of people accessing your RSS feed. Profit motive aside (there are bloggers who don't syndicate and who also don't have any ads up either) there's a basic interest in knowing how many people are reading your site. I found one tech-oriented blog that described a work-around, but it sounded rather difficult. (It involves coding your blog so that a unique code is assigned to each person who accesses your RSS feed so you can keep track.)

One other note: to Typepad's credit, they now encourage people to syndicate the entire post, not just an excerpt. Next to the syndication settings: "Most weblog readers who use aggreagators or news readers appreciate the option of being able to read an entire post in the tool itself." I'm not sure if they make (or are planning to make) a similar recommendation in MT.

By the way, what browser do you use?

Posted by: Remy at 16:17 EST, March 07, 2004

Josh Marshall/Talkingponitsmemo.com DOES have a RSS feed: http://www.talkingpointsmemo.com/index.xml

Posted by: DA at 12:23 EST, March 12, 2004

Thanks, DA!

Posted by: Felix at 12:38 EST, March 12, 2004

RSS for the kicker and for my personal site are MT defaults. we didn't custom develop them and insert the crappy.

that said, i don't have the technical know-how to make them better. if you have specific suggestions (and instructions for the coding-impaired) I will certainly pass them on to the appropriate parties.

Posted by: elizabeth spiers at 11:30 EST, March 15, 2004

The Talking Points Memo feed illustrates the problem that Felix describes. Sure, there's a feed, but it's close to useless if you want to keep up with what Josh is writing.

The belief is there's some benefit from me reading TPM in my browser rather than in my aggregator. What that means is I read less of TPM than I would otherwise.

The worst, absolutely worst, offender in this regard is Arts Journal. Terry Teachout's RSS feed only offers completely enigmatic headings with no teaser text.

Crooked Timber and Bonobo Land get the balance about right (if you want to restrict what's in the feed -- which I don't think you should) with well-nigh a full paragraph in the feed. That's a decent second best to full text.

Posted by: Lance Knobel at 13:07 EST, March 17, 2004

hey everyone- if you need an RSS template, i can send you ours- it's formatted to include the whole post. just shoot me an email.

Posted by: jake at 15:58 EST, March 17, 2004

But you know if someone's on MT you should just try all the different standards: /atom.xml, /rsd.xml, /index.rdf, and /index.xml. They'll all be differently formed and maybe you'll like one of them more than the one's publicized on the site. The day I reformulate MT's packaged RSS code is the day I've finished reading every book in the house and watching ever reality show I've Tivo'd.

Posted by: Choire at 0:51 EST, March 20, 2004

OK, consider me named n shamed. I'm with the program now. At least I think I am. I do have an RSS feed on my new site (I actually WAS on blogspot until now) but as I don't have an RSS reader yet I have no idea how my feed looks to those who do. Any feedback will be gladly accepted.

Posted by: dlr at 9:23 EST, March 25, 2004