9.2 Throughput and Load Sharing

only for RuBoard - do not distribute or recompile

9.2 Throughput and Load Sharing

A cache cluster with load sharing can improve both throughput and reliability. Throughput is increased because N caches can handle more traffic than just one. Reliability is increased because when one cache fails, the other cluster members absorb the increased load.

The really cheap way to implement load sharing is with the DNS. You simply list the IP addresses of all members under a single host name . Most DNS resolver libraries change the order of the addresses for each lookup. Thus, requests are sent to the caches in a round- robin fashion. This technique provides very rudimentary failover when one cache goes down. When a browser cannot connect to one of the IP addresses, it should try the next one in the list. Unfortunately, this can take a long time (up to two minutes), and browsers don't usually remember which addresses don't work.

A more robust approach, if you can afford it, is to use a layer four switch or dedicated load balancing product (see Table 5-1). These products have a number of load sharing algorithms (see Section 5.2.2) and generally do a better job of spreading the load among cluster members. Furthermore, users are less likely to experience delays or other problems if one member fails.

You may notice that I am intentionally avoiding the word balance. Some of the load sharing techniques may not distribute the load evenly among caches. The DNS-based approach is difficult to predict and control. One or two really busy clients could send all their requests to one of the caches, possibly overloading it. Layer four switches can be configured to partition requests based on IP addresses ”either the source or the destination. A busy client or popular server can cause one cache to receive many more requests than the others.

Random mapping between clients and caches also causes some problems for session tracking. A number of web services (origin servers) assume that all requests from a single user come from the same IP address. They get confused if different HTTP requests that belong to a single session come from multiple addresses. It seems that these origin servers are clustered as well. A layer four switch sits in front of them and distributes requests based on the client's IP address. When the user logs in, its authentication request is sent to one of the origin servers. This sets up some state information on that server so subsequent requests are allowed. But if the subsequent requests go to a different server, they are denied . To accommodate client-side clustering, the server-side layer four switch needs to be aware of the session information (i.e., cookies). This type of so-called layer seven switching is increasingly common for distributed servers. However, you may still encounter a service that uses address-based distribution and won't work with your cache cluster.

only for RuBoard - do not distribute or recompile