Content Distribution and Internetwork Caching

At some point in the not-so- distant future, moviegoers will walk into their living rooms, plop down in front of a Web-enabled device, and watch a movie off the Internet. Or so it will seem. In fact, the movie will be delivered from a local server in viewers ' homes , pushed there by a video server during the middle of the night.

Applications like this one are the promise of the new content distribution systems coming online. Combining elements of caching, content selection, and application development, these systems will enable profound, new, network-aware applications to make intelligent use of caches installed in the network. Little wonder that scads of vendors and providers are looking to jump into the space.

All this promise is a good reason to bone up on content distribution basics. Start off by understanding what people mean by caching, the precursor to content distribution. Then make sure you've got a good read on the benefits of caching; there are three reasons to go with caching, and not all are about boosting performance. Next, learn how caching works, and how content is kept up to snuff. Performance here is particularly important, and while many vendors might tout top- notch numbers , the cost of that performance may vary widely (see "The Price of Performance,"). Finally, find out why content distribution might be just the answer for certain applications.

Square One

There are two basic types of network caches, though each performs the same function: copying popular content from an originating server so that it can be accessed more quickly by the user . Prepackaged cache appliances are the first type, bundling hardware and software together. Today, these appliances are installed in the carrier or enterprise network, but already vendors are talking about diminutive versions of these appliances for the customer premises, loaded with some 20Gbytes of memory.

The second type is caching software, which is likely to stay targeted at enterprises , as it requires users to purchase the necessary server hardware and install the software themselves .

The primary benefit of both types of caching is better performance. Moving content closer to the user reduces the number of router hops it has to traverse, so performance improves . Also, just cutting the distance between user and content speeds up the connection. Finally, a local cache server may be less loaded and better designed for performance than the originating content server. These factors cut page-load times by one-half or one-third of the time needed to retrieve the page from the original server.

While numbers like those might be reason enough to implement caching, there are other critical benefits. For starters, cache servers protect against the problem of instant popularity. Major events such as the release of the Starr Report or an Internet broadcast of a Victoria Secret fashion show cause huge traffic surges. Masses of users request a particular Web page, which swamps the server. Opening the network with faster optical or local loop infrastructure will likely only exacerbate this problem. However, replicating this content to other servers or caches distributes the traffic load around the Internet, alleviating the server bottleneck.

ISPs also win with caching and content delivery by reducing their reliance on costly Internet links. According to the Internet Research Group (www.irgintl.com), bandwidth at the Internet's core costs $800 per Mbit/sec per month, yet users at the edge pay only around $50 per month for 1.5Mbit/sec Digital Subscriber Line (DSL) links. By using caching, ISPs keep more traffic local, using less core bandwidth and narrowing this gap.

Finally, caches can be deployed locally to provide better oversubscription ratios. When carriers design their networks, they use mathematical models to balance the number of users vs. the amount of provisioned capacity to support those users. Rarely is there a one-to-one correlation between those metrics. This is particularly true of cable modem and DSL service providers, where the head-end links can easily be swamped if all users start downloading at once. Cache servers offload traffic, effectively enlarging the upstream pipe and preventing local saturation.

Just The Basics

The key then is to shorten the distance between the user and the commonly requested contentnot a particularly new science. PC design has long used fast memory caches to feed high-speed processors. Similarly, Web browsers use local-disk caching to improve local Web performance. This is why jumping back to a previous Web page gives the impression of much faster performance.

Network caches work on the same basic principles, but on a much grander scale. Unlike a private cache in the Web browser, however, the network caches select content based on the activity of hundreds and thousands of users.

Cache servers learn these patterns by intercepting user requests in one of two ways: transparent cache servers or proxy cache servers. Transparent cache servers sift through all traffic and hence require no modification to the end client. The key is that these cache servers see all of the traffic destined for the Internet. This means locating the cache server at a choke point in the network, like in front of a router connecting to the Internet. Alternatively, requests can be diverted to the server using a device like a layer-4 switch.

Then the cache server checks to see if the content is stored locally or not. If the content is already stored locally, it is delivered to the user. If not, the content is retrieved from the origin server, sent to the user, and possibly cached for future reference.

With proxy cache servers, network managers configure users' browsers to direct content requests directly to the cache. The proxy cache server then requests the content on behalf of the user. On the one hand, this lets network managers restrict the sites a user can visit. At the same time, though, this approach is obviously far more complicated, as it requires configuring each client. What's more, if the proxy cache fails, users cannot access the Web (see Figure 1).

Figure 1: There are two types of cache servers on the market today. Transparent cache servers require no changes to the client. They sift through the Internet-bound traffic and either deliver stored content from cache or send the request on to the content server. Proxy caches require changing the client to receive all Internet requests. Content that is not stored in the cache is then retrieved for the user from the originating content server.

Guaranteed Fresh

Simple enough, but how does a cache know when to discard content? Today's cache servers use either passive or active caching. With passive caching, caches check with the originating server about the freshness of the content, using the get if modified command in HTTP. Normally, a cache server uses the get command to request an object from a content sever. However, if the object is already stored, the cache server uses the get if modified command, which only pulls down the object if it has been modified since the last request. Then the cache server can compare the modification dates of the object from the server and the one in the cache, and serve up the newer one to the user.

The problem with passive caching is performance: Users have to wait for the cache server to check every request. Passive caching can be improved by checking freshness data, such as the time expiration date, in the object's header. When the object reaches a specified age, the cache server fetches fresh content.

Active caching improves caching performance by using heuristics to assess the life expectancy of an object. The server can calculate when an object entered the cache, how long it's been held, the origin IP address, or myriad other factors.

The big benefit here is that each request doesn't have to be checked. Instead, the cache server makes certain assumptions about the time to live for an objectsay, two days, for example. During that interval, all requests for the object are serviced immediately from cache. Once those two days pass, the cache refreshes the object.

Content Delivery

Content delivery is in effect another approach for keeping content fresh. While caching waits for customers to request information, content delivery lets companies proactively push the information into caches close to the user.

Here's how content delivery works. First, content providers designate content to be replicated to network caches. Here, the network manager either provides a list of elements to be replicated by the content delivery provider or uses tools that troll through the site, marking the elements to be replicated. The content provider then replicates those elements around the globe. Content consumers are then directed to the nearest server through DNS.

Geographic penetration of these caches is critical to the success of the content delivery system. The more caches deployed, the greater the likelihood that users will be able to retrieve a document or element from a cache located geographically nearby.

The sheer number of caches, however, isn't the only critical factor. A successful content delivery provider needs a high density of geographic coverage. With more caches in a given area, there's a better chance of finding a cache near the user. For example, Akamai claims to have 4,200 caches across 50 countries around the globe.

Many content delivery providers differentiate themselves by how they bring the content to the customer. Most replicate content across the Internet, but this can result in all sorts of performance problems. Some providers take a more costly approach, delivering their content via satellite into nodes comprised of satellite receivers and caches connected to the Internet. One start-up, Fireclick, preloads objects directly into a user's cache. Although this may well improve performance, users are likely to balk at the idea.

Many content distribution systems are also promulgating their own APIs to encourage developers to create network-aware software. Like the video server pushing a preselected movie into the user's cache (see Figure 2), or a main office distributing training videos into the caches of the field offices, these applications can be more intelligent about how they utilize network resources.

Figure 2: By writing custom APIs, applications such as video retrieval can be more intelligent, improving the user experience. First, the personal cache server shares the user's movie preferences with the video server (1). Then, during off-hours, the video server transfers movies likely to be watched by the user into the user's personal cache (2).

The result is a content distribution system that begins to look like an interactive and predictive television system. Users can request certain types of content through a Web browser and have it loaded into the cache. Even when users do not request the content, the different elements of the delivery network can work together to anticipate user needs. The cache server can monitor user preferences and send feedback to the content provider. The service can then issue the request to the content server to distribute the most appropriate material at the optimal part of the day.

The Price Of Performance

It should be pretty clear by now that keeping up with content requests is no mean feat. While the number of network caches available on the market has exploded, the price/performance ratio remains a critical issue.

Cache server vendors talk about content delivery performance in terms of the object ratio and byte hit ratios. The object ratio compares the average number of objects sent from cached data to the total number of objects served from cache. The byte hit ratio compares the average size of those objects served from cache against those objects delivered.

Beefing up cache performance to improve either of these factors is usually a matter of adding more memory and hardware. Your best bet is to emphasize memory, as it can be much faster than retrieving data from a disk. However, many vendors get great performance numbers by going a step further and clustering servers together. While this can boost performance, it can also dramatically increase the price tag.

Today, two protocols enable cache clustering: the Internet Caching Protocol (ICP) and the Caching Array Protocol (CARP).

With ICP , cache servers can pool their resources to detect if objects have been cached in one of the other servers. However, this approach requires additional instructions, which can affect performance. Cache server vendors can avoid the performance issues by using layer switches to link the servers, which in turn can cost hundreds of thousands of dollars each.

CARP solves the problem by running a hashing algorithm on the main network cache to determine the network caches in the array that should receive the URL request. The only snag is that the hardware can't forward network traffic as fast as a switch.

Resources

There are a number of useful caching resources on the Internet: www.caching.com provides a good overview of how caching functions; www.web-caching.com also provides a great site for introducing caching and listing vendors that offer content distribution services, as well as offering numerous links to research, books, and other sites.

Speaking of books, consider Web Proxy Servers by Ari Luotonen, published by Prentice Hall. Duane Wessels will be releasing a book appropriately titled, Web Caching , published by O'Reilly and Associates in January 2001.

If implementations are what you're after, there are loads of vendors pushing caching and content delivery software. If open-source software is your beat, consider www.squid-cache.org, the site for Squid open -source caching software. It might not be the most scalable code out there, but it's free. For more info on Squid, ask the folks at IRCache (www.ircache.net); they've got Squid's original developer on staff, they run bakeoffs, and they do all sorts of workshops on caching.

This tutorial, number 148, by David Greenfield, was originally published in the November 2000 issue of Network Magazine.