State Maintenance Techniques

only for RuBoard - do not distribute or recompile

Several techniques have been developed to cope with the stateless nature of HTTP so that applications can provide a measure of continuity across requests. Some of these methods use the client for storing the information; others keep the information on the server.

Client-Side State Maintenance

For client-side techniques, the state information is maintained by the client and submitted to the server. These methods are described here, using the online quiz scenario to illustrate how each one might be used to keep track of state information:

Hidden fields enable you to embed information in a form that is sent to the client, where it remains stored in the page displayed by the browser until the user transmits it back to the Web server. For the quiz, you can generate a form that includes hidden fields that indicate the current numbers of correct and incorrect responses received so far. Each time the user submits an answer to a question, you update the totals and include the new values in the next form sent to the user s browser.
You can include state information as part of a URL. For example, the action parameter for each form that presents a quiz question could include in the URL the number of correct and incorrect responses received from the user for the previous questions. If you specify that the form contents should be returned as a GET request, then when the user answers the current question and submits the form, the browser will encode the values of fields in the form and add them to the URL. For example, the URL that returns the response for the twenty-second question might look something like this after the browser adds a parameter to indicate the user s answer:

http://www.snake.net/cgi-perl/quiz.pl?correct=18;incorrect=3;answer=b
Cookies are a client-side mechanism invented specifically to allow state maintenance.^[1] Cookies are transmitted back and forth in the headers of requests and responses exchanged by the client and server. When the server wants the browser to store a value, it sends the information using a Set-Cookie: header. For example, the quiz scores could be initialized by sending the client a Set-Cookie: header like this one along with the page that presents the first question:

^[1] If a worse name than cookie was ever invented for nomenclature purposes, I don t know what it is.
```
 Set-Cookie: quiz-scores=incorrect&0&correct&0 
```
Each time the user submits an answer to a question, the browser returns the quiz-scores cookie in the request headers using a Cookie: header. The server extracts the cookie value, updates the totals, and sends back a new Set-Cookie: header with the next page so that the browser can update the quiz-scores values it is storing. For more information about the cookie specification, see Appendix B, References and Further Reading.

For each of these techniques, the server sends the client the initial state information to be stored. The client returns this information with each subsequent request in the session, possibly receiving in return updated information from the server after the server processes the request. The server need never remember the state values because they are stored on the client. It needs to know only how to handle information received from the client (which includes the current state data and the new response). As the server processes the request, it updates the state information and returns it to the client, where it remains until the client submits the next request.

Because information is stored by the client when you use these techniques, they require no reconfiguration of the Web server. However, client-side state maintenance is subject to certain drawbacks:

You might accumulate large amounts of information, all of which is sent back and forth for each request. If you implement a shopping cart that stores the currently chosen items on the client end, the amount of information grows with each new selection. This can be a problem, because if you store information in the URL or using hidden fields in a form that s sent to the server using the GET method rather than POST, you may find the information being truncated silently. Cookies often have a size limitation as well (about 4KB in many implementations).
Some kinds of information really shouldn t be sent back to the client at all, such as personal or private information. Suppose you require a user to supply a password to gain access to a form for editing a preferences profile. If you put the password in a hidden field so that the user need not enter it again when the editing form is submitted, you expose the password to additional snooping as it makes another round trip over the network.
Information stored on the client end is subject to modification by the user and therefore not necessarily trustworthy. If you include a value in the state data that indicates the user s access privilege level for the information in your database, the user could change the value to a higher level by editing this information. Hidden fields can be seen easily using the browser s View Source command, and the form contents can be modified easily. (The method is described in Chapter 9.) Cookies can be viewed and modified by using an editor to open the cookie file created by the browser. The more data you store on the client side, the worse the problem becomes.
Use of cookies is subject to variation in client capabilities. You can t use them to handle requests from a client browser that is so old it doesn t support the cookie protocol. Cookies also require cooperation on the part of the user. Browsers that can handle cookies typically enable the user to indicate whether to receive them; many users are suspicious of cookies and refuse to accept them.
Information stored on the client side can be extremely transient. If the user closes the browser window in the middle of a session, any state information stored in hidden fields is lost and the user must start over. The same is true for state information included in URLs, with the exception that it s possible to set a bookmark to retain the link. (This can be good or bad. One use for bookmarking is to allow the user to suspend a session and return to it later. On the other hand, if the user sets a bookmark to a page generated by an application that has time-limited sessions, and then returns to the bookmarked page only to find the session has expired, the result is frustration over the now-invalid link.) Normally, if you want to maintain state information for an extended time on the client side, hidden fields and URLs are unsuitable. Cookies work better for long-term data because they can be given a specific lifetime.

Despite these drawbacks, client-side techniques are easy to use, and you ll probably find them perfectly suited for some applications, especially simpler ones. For applications where client-side state storage isn t good enough, however, it s necessary to manage state data on the server.

Server-Side State Maintenance

An alternative to client-side techniques is to store state information on the server side. In this case, the client announces itself to the server on each request using some kind of client identifier, and the server associates the ID with the appropriate state information that it maintains on the client s behalf. Some techniques you can use for serverside state maintenance are as follows:

You can take advantage of the existence of the Web server as a long-running process. If your Web server allows it, you can ask it to store information for you in its own memory, and then access that information on subsequent requests.
You can store state information in more traditional repositories such as a file or a database.

Use of server memory is applicable primarily for storing small amounts of information; it s not a good technique when you need to maintain potentially large amounts of data. If your storage requirements are significant, files or databases offer much greater capacity.

Server memory also is unsuitable if you need durable or persistent information that exists until you delete it explicitly. Under Windows, for example, Apache runs as a single process, and server memory is lost each time the server is restarted. Under UNIX, Apache runs using a multiple-process architecture involving several children. This means you can t rely on getting the same child for each request issued during a session, but the children can use shared memory to communicate with each other. (You can use the Perl IPC::Shareable module to implement a shared storage area.) State information stored this way survives the termination of any given child process, but still is lost across machine restarts. If you need persistence of state data, you should use files or a database.

Several characteristics of server-side state maintenance techniques distinguish them from client-side methods:

With client-side state storage, the client provides all the state data on each request, so requests are self-identifying. When you store information on the server side, you must have some way to uniquely identify each client so that you can associate requests from a given client with the appropriate state data.
Server-side storage reduces the amount of information sent back and forth. For session-tracking purposes, the client need not provide all state information on each request, just a session ID and whatever new information is specific to a given request. In a shopping cart, for example, the client need only transmit each item as it is selected, not the entire contents of the cart.
Server-side storage is more secure, because clients cannot tamper with it; they cannot see or modify state information except what you allow to be seen or modified. It s also more secure in terms of exposure to other users; the fewer times you send information back and forth over the network, the better. (Of course, if you must be really secure against snooping, you should use an encrypted connection.)
Server-side state management requires more complex server support. If you want to use a file as a state container, you must be concerned about using locking protocols to keep multiple processes from writing to the file at the same time. It s also necessary to institute storage and retrieval procedures. A database eliminates these problems, because it handles locking issues for you, and storage and retrieval is a matter of writing the correct SQL statements. But of course, if you want to use a database to store state information, you must set one up first. In either case, state maintenance consumes more processing, memory, and disk resources on the server host compared to client-side methods.
If you want permanent storage, server-side techniques make this easy because you can put the information in a file or a database. On the other hand, if you don t want a session record to be permanent, you have to provide a mechanism for identifying and deleting expired sessions. Expiration of state information is easy for client-side methods, because the server need not keep track of when to dispose of it. With hidden fields or URLs, information expires automatically if the user closes the browser window or moves on to visit another site. With cookies, you have two options: Set an expiration date in the cookie and let the browser delete it when that time arrives; or specify no explicit expiration date, in which case the browser deletes the cookie when it exits.
Server-side techniques may result in orphan sessions sessions that users begin but never complete. A person who starts adding items to their shopping cart at your online store and then decides to leave without buying anything creates an orphan session. Here, too, you must be concerned about identifying and expiring sessions that are no longer of value.
Server-side storage enables you to do some things you just cannot do with client-side techniques, at least not without some horrendous hackery. One of these is coordination of state data among multiple clients. If you re trying to implement a game that enables people to play each other over the Internet, client-side storage requires you to store state in multiple clients, leading to synchronization problems. Server-side methods enable you to centralize state information so that multiple clients can access it more easily. You can also synthesize shared resources using state data gathered from multiple clients, such as a high scores list based on all previous player sessions.

The preceding comparison is summarized in Table 8.1. In general, it s more work to manage state information on the server side, but it s also more secure and reliable. We ll concentrate on server-side methods for the rest of this chapter. Of these techniques, storage of state information in the Web server s memory doesn t provide persistence, so we won t discuss that further. That leaves storage of state data in files or a database. If you don t want to worry about locking protocols or inventing storage and retrieval conventions, however, that rules out files. The only remaining choice for server-side management of state information is to store it in a database. It is, of course, a complete coincidence that we arrive at this conclusion in a book about MySQL! Funny how that works.

Table 8.1. Comparison of Client-Side and Server-Side Session Information Storage Methods
	Client-Side Methods			Server-Side Methods
	*Hidden Field*	*URL Parameter*	*Cookie*	*Server Memory*	*File*	*Database*
Suitable for Long-Term Storage			x		x	x
Survives Server Restarts	x	x	x	Perhaps	x	x
Survives Host Restarts	x	x	x		x	x
Automatic Expiration	x	x	x
Hackable by Client	x	x	x
Potential for Exposure to Other Clients	High	High	High	Low	Low	Low

Client Identification

To use server-side state maintenance, you must be able to uniquely identify each client. Otherwise, you have no way of grouping requests from a given client. But just how do you identify the client? The problem is a difficult one, because browsers don t provide any information that is guaranteed to be both unique and persistent. This section describes some techniques you can use to identify clients, classified into two general categories: passive and active. Passive methods use some form of information that is intrinsic to the client itself, such as the IP number of the host from which the request is received. For active methods, the server generates an ID and sends it to the client when the session begins, and the client returns the ID with each subsequent request until the session ends.

Passive Client Identification Methods

I m going to describe passive identification techniques briefly, but only for the purpose of disparaging them. The main point you should take away from this discussion is that passive identification is unreliable.

One way to identify clients passively is to use their IP number. When the Web server sets up the CGI environment for a script, some of that information characterizes the client. For example, the REMOTE_ADDR variable indicates the client host s IP number. It s tempting to use this variable in your scripts to distinguish one client from another, but unfortunately client IP numbers are not necessarily unique or stable:

On a host such as a UNIX box that allows multiple users to log in, many users can run browsers at the same time. Requests from all these users originate from the same IP address, which therefore does not serve as a unique client identifier.
Some Internet services route requests from their users through a proxy server. All such requests appear to come from the same host, regardless of the real point of origin. The same is true for sites that use NAT or other forms of address translation to map multiple client machines onto the same IP number. (This is an increasingly common situation in homes and small offices that have multiple machines but pass all traffic through a router that serves as the single point of connection to the Internet.)
It s possible for a client s IP number to change periodically, causing any IP-based check to fail. Client hosts that obtain their IP number from a DHCP server may undergo address changes each time their IP lease expires and they obtain a new number from the server. If a service runs multiple proxy servers that handle users in round-robin fashion, a client s IP address changes as it cycles through these servers.

The same limitations also apply to client identification attempts based on host names, which are closely tied to IP numbers.

Is IP-Based Client Identification Completely Worthless?

IP numbers and host names are of no use for applications that serve the Internet at large. However, they may be of some value in more highly constrained situations. If your organization exercises complete control over an intranet consisting entirely of single-user hosts with static IP addresses, for example, it may be reasonable to assume for identification purposes that each client s IP number or host name is unique and provides a reliable indicator of request origin.

Another way to attempt passive identification is to use the time of a client s initial connection. Each moment in time is distinct from every other, after all. Unfortunately, that property of the natural universe does not translate well into the world of computers that divide time into discrete slices. There s no way to guarantee that two requests won t arrive so close together as to occur during the same time slice. If this happens, two clients assume the same identity. A further problem with time-of-connection identification is that it allows an imposter to claim my initial connection was at time t , and gain access to any session that happened to begin at that time.

Conclusion: Passive techniques for identifying clients don t work very well.

Active Client Identification Methods

The alternative to passive client identification is to play an active role on the server end. The procedure goes like this:

When a client begins a session, generate an identifier on the server side and remember it for later so that you can associate it with the state data you re maintaining for the session.
Give the identifier to the client and require that the client return it with all subsequent requests that are part of the session.
Each time a request arrives that contains the identifier, look up the associated session and interpret the request in the context of the session s current state.
When the session ends, forget the identifier or mark it inactive to cause any additional use of the identifier by the client to fail.

One way to assign client identifiers to new sessions is to use a sequence generator such as an AUTO_INCREMENT column. This is easy to implement, because MySQL manages for you the details of making sure each request for an identifier is satisfied with a unique value. Another active identification method is to require users to begin a session by providing a username and password. If the name and password check out okay, you can use the name as the client ID value. But do these techniques provide good client identifiers? To answer this question, we must ask what characteristics distinguish good client identifiers from bad ones. Here are a couple that are important:

Each identifier should be unique, so that you don t mix up clients with each other.
Identifiers should not be guessable. You want the ID that you give to a client to be used only by that client, not by someone else who is trying to hijack sessions and just happens to be able to guess a valid identifier. Basically, this means your identifiers should be random and chosen from a large pool of possibilities.

When assessed against these characteristics, how do AUTO_INCREMENT values and login names stack up as client identifiers? In both cases, the values are unique, so they satisfy the first criterion of good identifiers. However, both methods involve easily guessable values. AUTO_INCREMENT values form a linear sequence, so they re extremely predictable. Usernames don t form a sequence, but they re still much more easily guessed than random values. If you re using a login phase in an application, it s a good idea to generate a random value after a successful login and use that rather than the username for the client identifier.