B.4 Experimental Features

only for RuBoard - do not distribute or recompile

B.4 Experimental Features

Since the publication of RFC 2186, a number of experimental opcodes and flags have been added to ICP. These were generally added so specific software developers could implement new features for their products.

B.4.1 Pointers

ICP_OP_MISS_POINTER (18)

ICP_FLAG_POINTER (0x20000000)

Mirror Image Internet designed a web caching product that operates in a cluster configuration. Each node in the cluster knows , or remembers, which objects are stored by each neighbor. Thus, instead of returning an ICP miss message, a cache may be able to say, "I don't have this object, but I know who does." In other words, a cache provides a pointer to an object located in a neighbor cache. Mirror Image has proposed some experimental ICP opcodes and features to implement these features.

To indicate its support for the pointer features, an ICP client sets the ICP_FLAG_POINTER bit (0x20000000) in the Options field of a query message. An ICP server then sends a MISS_POINTER message instead of a MISS if the option bit is set and it knows of a neighbor cache that holds the requested object. The payload of the MISS_POINTER message is a list of IPv4 addresses, one for each neighbor cache that has the object. The payload does not include the URL because the ICP client should be able to determine the URL from the ICP reply's Reqnum field.

ICP pointers and other types of distributed cache directories do not address permission and authorization issues. A directory may tell client A that cache B has a cached copy of the response. However, this does not necessarily mean that A is allowed to request it from B. If A and B have no prior agreement in place, then most likely B would deny any request from A. In the case of a MISS_POINTER reply with many addresses, it is entirely possible that the ICP client does not have permission to get the object from any of them.

Researchers at AT&T also developed a centralized directory system, which they call CRISP [Gadde, Chase and Rabinovich, 1998]. Their implementation, based on Squid, uses ICP for information exchange but does not use the opcodes described here.

B.4.2 Object Advertisement

ICP_OP_ADVERTISE (19)

ICP_OP_UNADVERTISE (20)

ICP_FLAG_PREADVERTISE (0x10000000)

A centralized directory requires some method for the group members to inform the directory when objects are added or removed. The two experimental opcodes, ICP_OP_ADVERTISE and ICP_OP_UNADVERTISE, provide this functionality. The payload of an ADVERTISE message is simply the URL, and the recipient does not generate a reply.

When a parent cache also provides a directory service, the ADVERTISE message normally follows a QUERY. For example, a child cache sends a QUERY to its parent, which sends back a MISS. The client fetches the URL from somewhere, then sends an ADVERTISE message to the parent. As an optimization, we can eliminate the advertise message by setting the ICP_FLAG_PREADVERTISE bit in the QUERY message. The flag indicates that the sender intends to have the named object in its cache soon.

B.4.3 Request Notification

ICP_OP_NOTIFY (12)

The ICP_OP_NOTIFY opcode and associated flags were added to support prefetching between Squid and the Wcol proxy. ^[A] Wcol can be used to prefetch objects through a normal (nonprefetching) proxy, as shown in Figure B-5. Instead of sending client requests through Wcol directly, the NOTIFY message is used to tell Wcol that someone requested a particular resource.

^[A] You can find more information on Wcol at http://shika.aist-nara.ac.jp/products/wcol/wcol.html.

The Squid cache sends a NOTIFY message to the prefetching engine (Wcol) for every client request it receives. The prefetching engine may also elect to request the given URL and parse it for inline images and referenced links. The format of the NOTIFY message is identical to the QUERY message. However, the receiver does not send any reply message back. The following four flags are defined for the Option Data field to provide additional information about the requested URL:

ICP_NOTIFY_MISS (0x00000008)

The requested URL does not exist in the cache.

ICP_NOTIFY_HIT (0x00000004)

The requested URL exists in the cache.

ICP_NOTIFY_REFRESH (0x00000002)

The client's request includes a no-cache directive.

ICP_NOTIFY_IMS (0x00000001)

The request includes an If-Modified-Since header or other cache validator.

Although NOTIFY was invented to support prefetching, it is not limited to that. It can also be used for simple remote request logging, for example.

Figure B-5. ICP NOTIFY usage

B.4.4 Object Removal and Invalidation

ICP_OP_INVALIDATE (13)

ICP_OP_PURGE (14)

In a tightly coupled cache cluster, it may be desirable to propagate object deletions and invalidations with ICP. The INVALIDATE message requests a cache to mark the named object as stale, without removing it. The next time the object is requested, it should be validated with the origin server. Similarly, the PURGE message requests that a cache remove the named object from its storage. There is no response for either of these messages. Because ICP messages are normally sent via UDP, there is no guarantee the message will be received. Even if the message is received, there is no guarantee the other cache will honor the request.

Remember that ICP provides no authentication mechanism. In theory, this feature could allow a third party to wipe out entirely a cache's contents. An implementation should probably accept these requests only from specific IP addresses. Even then, the address may be spoofed or the machine broken into. It may also be a good idea to limit the number or rate of delete/invalidate messages that are accepted.

B.4.5 MD5 Object Keys

ICP_FLAG_MD5_KEY (0x08000000)

At some point in time, product developers considered using MD5 checksums, instead of URLs, in ICP messages. Although nothing was ever implemented (as far as I know), the ICP_FLAG_MD5_KEY bit was reserved. Note that there is no standard for calculating an MD5 checksum for an HTTP request or response.

B.4.6 Eliminating URLs from Replies

ICP_FLAG_DONT_NEED_URL (0x04000000)

The ICP_FLAG_DONT_NEED_URL option bit informs an ICP server that it doesn't need to include the URL in its reply message. This optimization reduces the size of ICP replies by about 75%.

B.4.7 Wiretapping

ICP_OP_WIRETAP (15)

The ICP_OP_WIRETAP message is a request to receive a stream of NOTIFY, ADVERTISE, and UNADVERTISE messages from a neighbor for a certain amount of time. This opcode was added for Cidera, which uses these ICP messages in its satellite-based cache filling service. Rather than statically configuring a cache to continually transmit these messages to its collection service, it wanted something more flexible and dynamic. The WIRETAP payload holds the amount of time, in seconds, for which the recipient should send the NOTIFY and ADVERTISE messages back to the sender. The time is encoded as a single, unsigned, 8-bit value. Thus, the WIRETAP message must be sent at least every 255 seconds, or the flow of messages will stop.

B.4.8 Prefetching

ICP_FLAG_PREFETCH (0x02000000)

The ICP_FLAG_PREFETCH option, also added for Cidera, is a request for the recipient to prefetch the named URL if it is not already cached. This is a suggestion only; the recipient may refuse to prefetch the URL without giving a reason.

only for RuBoard - do not distribute or recompile