9.2. Publish and Subscribe
The traditional guidelines for the use of syndication feeds present a small problem. There is always going to be a balance between the need to be up to date and the need to refrain from abusing the feed publisher's server by requesting the feed every few seconds. While the norm is to request the feed a maximum of once an hour, many feeds deserve to be followed much more closely. Conversely, many feeds update only once a day, or less often. Requesting those once an hour is a waste of time and resources and potentially expensive for the publisher. On top of the compressed feeds and conditional GET already discussed, one other idea has long existed within the feed world: Publish and Subscribe.
Let's think of the earlier situation. We have a feed, and this feed has its users. The users take the feed and do what they willI've presented some examples of potential uses in the previous chaptersbut each user depends on his copy of the feed (whether in memory, or converted to another format and saved) being up to date. In order for this to happen, the users subscribe to a system that watches the feed continuously. This system then publishes notifications to all of the users when the feed changes. The users are then responsible for updating their copies of the feed, ordinarily by requesting it from the server.
Currently, there is no Publish and Subscribe mechanism for Atom, but there are two ways to implement such a system within RSS: one with the relevant elements included in RSS 2.0, and the other suggested by an RSS 1.0 module. We will deal with these in turn.
9.2.1. Publish and Subscribe Within RSS 2.0
RSS 2.0's Publish and Subscribe system is the result of work by Userland Software and its CEO Dave Winer. It was introduced with, and perhaps inspired, the release of RSS 0.92 and features heavily in Userland products. Furthermore, its workings are based on XML-RPC and SOAP, two protocols that Winer was instrumental in starting. But that is not to say that RSS 2.0's Publish and Subscribe is in any way proprietaryit is not.
There are three characters to watch in this drama: the user, the feed, and the Publish and Subscribe system, itself better known as the cloud. Here is the process:
Note that by design, the RSS 2.0 Publish and Subscribe system expires subscriptions after 25 hours, forcing them to be renewed every day.
As I described in Chapter 4, the RSS 2.0 Publish and Subscribe system is denoted in the RSS feed with the cloud element:
<cloud domain=" Domain or IP address of the cloud" port="TCP port on which the cloud is listening" path="The path to the cloud's listener" registerProcedure=" The procedure name to register with the cloud" protocol="Either xml-rpc or soap, Case-sensitive" />
You see that we talk of passing messages via XML-RPC or SOAP? These may require some explanation. The two protocols are the basis of the fashionable technology known collectively as web services. We have already used some web services in previous chapters: getting information from Google using SOAP, for example. With those examples, we were passing a query, encoded in XML, and getting a set of results back, also encoded in XML. In the case of RSS 2.0 Publish and Subscribe, we are doing something similar: passing XML-encoded messages between systems. What makes the RSS 2.0 Publish and Subscribe system different from the other uses of web services you have already seen is that the user's system has to not only pass a message and wait for a reply, but also continually listen for other systems trying to talk to it. For this reason, it cannot be used with machines that reside behind Network Address Translation (NAT) systems. The user's machine must be directly addressable from the rest of the Internet, and it must be listening for people trying to do so.
RSS 2.0 Publish and Subscribe has been publicly implemented at least twice. Userland Software incorporates it directly into its own products, and NewsIsFree (http://www.newsisfree.com), the RSS aggregator site run by Mike Krus, uses it as well. Both services produce feeds with a cloud element, and both run servers that will notify you of changes to the feeds.
These are both nice services, but what if you want to use your own Publish and Subscribe system to update, say, a web site's rendering of a feed? No problem: we'll just roll our own Publish and Subscribe system (see the Section 9.3).
9.2.2. Publish and Subscribe with RSS 1.0
As it stands, the RSS 2.0 version of Publish and Subscribe is currently unavailable to standard RSS 1.0 users; it exists neither in the core specification nor in any published modules. But that is not to say that you cannot create a module to import the cloud element. You are, of course, at liberty to do so.
Meanwhile, the rest of us may be looking at the mod_changedpage module. This Publish and Subscribe module, written by Aaron Swartz, is fundamentally different from the system used with RSS 2.0. Where the latter uses web services protocols to communicate between systems, mod_changedpage uses simple HTTP POST procedures to handle the same job.
It works like this: the RSS 1.0 feed's channel contains the single element cp:server, which contains simply the URL of the Publish and Subscribe service for the feed. A user who wants to subscribe to notifications for this feed sends an HTTP POST request to that URL, taking two parameters:
Note that the :// of the URLs within the parameters are entity-encoded as %3A%2F%2F.
The original proposal document for this specification states:
Either way, the URL is received at the Publish and Subscribe service, which returns a HTTP status code of 200 if the subscription is successful and an HTTP status code of 400 if the subscription failed (it is also recommended that a 400 code should be accompanied by a plain-text file explaining the error as thoroughly as possible).
When the feed updates, the Publish and Subscribe system sends a similar HTTP POST request, with one parameterthe URL of the changed feed:
The user's system can then do what it likes with that feed.