16.3 Securely Using Fields, Hidden Fields, and Cookies

only for RuBoard - do not distribute or recompile

One of the reasons that it can be difficult to develop secure web applications has to do with the very architecture of web applications. When you develop an application, you generally write a body of code that runs locally on the web server and a much smaller body of code that is downloaded and run remotely on the user's web browser. You might spend a lot of time making sure that these two code bases work properly together. For example, it's very important to make sure that the field names downloaded in web forms exactly match the field names that server-side scripts are expecting. And you will probably spend time making sure that the HTML forms, JavaScript, and other codes that are downloaded to the browser work properly on a wide range of different browser programs.

Even in the best of times, it can be difficult to get software on the web browser and the web server to properly synchronize and interoperate. What makes this whole process difficult from the security perspective is that attackers, by definition, don't play by the rules. Sure, they can run your HTML forms and JavaScript in well-behaved browsers, but they can also pick apart the code, analyze it, and send completely made-up responses back to your web server. These sorts of attacks are difficult to detect because they are very hard for normal web developers to test against after all, most web developers don't have a stable of CGI-script attack tools.

There is nothing inherently wrong with storing this information on the web browser instead of the web server; indeed, storing this information on the browser eliminates the need for a backend database, user tracking, and a lot of other technology. But if you store information on the user's web browser, you must validate this information when it is passed back to the web server to make sure that it has not been modified.

Many programmers do not realize the need to validate information returned from the web browser to the server. For example, in December 1999 engineers at Internet Security Systems (ISS) discovered that many e-commerce scripts from different vendors all shared a common vulnerability: they maintained the shopping cart, complete with the price for each item, on the user's web browser without using any form of validation.^[4] When an invoice was prepared and a credit card charged, they blindly trusted the prices provided by the shopping carts. Thus, any attacker who wanted to give himself a discount could simply go shopping, save the server's HTML onto his hard drive, edit the prices, and then click on the "Buy" button.

^[4] ISS reported the security problem to the 11 vendors in December 1999, then released the information about the vulnerability to the press in February 2000. For further information, see http://www.cnn.com/2000/TECH/computing/02/04/shop.glitch.idg/.

In a Spring 2001 study,^[5] four MIT graduate students discovered that many e-commerce sites did not properly validate the information in cookies. As a result, Sit and Fu were able to make subtle modifications in the cookies at e-commerce sites and gain access to unauthorized information.

^[5] See "Dos and Don'ts of Client Authentication on the Web," USENIX and MIT Technical Report 818, by Kevin Fu, Emil Sit, Kendra Smith, and Nick Feamster.

16.3.1 Using Fields Securely

When checking arguments in your program, pay special attention to the following:

Filter the contents of every field, selecting the characters that are appropriate for each response. For example, if a field is supposed to be a credit card number, select out the characters 0-9 and leave all other characters behind. This will also allow people to enter their credit card numbers with spaces or dashes.
After you filter, check the length of every argument. If the length is incorrect, do not proceed, but instead generate an error.
If you use a selection list, make certain that the value provided by the user was one of the legal values. Attackers can provide any value that they wish: they are not constrained by the allowable values in the selection list.
Even if your forms use JavaScript to validate the contents of a form before it is submitted, be sure that you revalidate the contents on the server. An attacker can easily turn off JavaScript or bypass it entirely.

16.3.2 Hidden Fields and Compound URLs

A hidden field is a field that the web server sends to the web browser that is not displayed on the user's web page. Instead, the field merely sits in the browser's memory. When the form on the page is sent back to the server, the field and its contents are sent back.

Some web developers use hidden fields to store information that is used for session tracking on e-commerce systems. For example, instead of using HTTP Basic Authentication, developers sometimes embed the username and password provided by the user as hidden fields in all future form entries:

<INPUT TYPE="hidden" NAME="username" VALUE="simsong"> <INPUT TYPE="hidden" NAME="password" VALUE="myauth11">

Hidden fields can also be used to implement a shopping cart:

<INPUT TYPE="hidden" NAME="items" VALUE="3"> <INPUT TYPE="hidden" NAME="item1" VALUE="Book of Secrets:$4.99"> <INPUT TYPE="hidden" NAME="item2" VALUE="Nasty Software:$45.32"> <INPUT TYPE="hidden" NAME="item3" VALUE="Helping Hand:$32.23">

Instead of embedding this information in hidden fields, it can be placed directly in the URL. These URLs will then be interpreted as if they were forms that were posted using the HTTP GET protocol. For example, this URL embeds a username and password:

http://www.vineyard.net/cgi-bin/password_tester?username=simsong&password=myauth11

It's quite easy to use hidden fields. Little or no information needs to be stored on the server. And unlike cookies, which are limited to 4096 bytes, hidden fields can be practically any length whatsoever.

There are problems with using hidden fields in this way, however:

If the user presses the "Back" button, items may be removed from the shopping cart. Sometimes this is the desired behavior, but usually it is not.
HTML pages used by one person might be viewed by other people, possibly because the computer is shared. In this circumstance, the first user's username, password, or shopping cart contents might be disclosed.
If you use URLs to embed information, the complete URL including the embedded information will be stored in the web server's log files. The full URL may also be passed by the user's browser in the referrer [sic] header when the user accesses another web server. This may compromise the user's privacy and/or security.
In the vast majority of cases, the contents of the hidden field received by the web server are identical to what was originally provided. But there is no guarantee. An attacker can save your HTML to a file, analyze the form, and issue his own HTTP GET or POST command with whatever contents he desires. An attacker can also submit the same web page over and over, with slight modifications, probing for vulnerabilities. There is no way to stop this sort of behavior, so you must defend against it.

There's no way to correct the problem with the "Back" button, but the second and third problems can be handled using cryptography, as described in Section 16.3.4.

16.3.3 Using Cookies

One attractive alternative to using hidden fields or URLs is to store information such as usernames, passwords, shopping cart contents, and so on, in HTTP cookies.

Users can modify their cookies, so cookies used for user tracking, shopping carts, and other types of e-commerce applications have all of the same problems described for hidden fields or compound URLs. But cookies also have problems all their own, including:

Old cookies may continue to be used, even after they have "expired."
Users may make long-term copies of cookies that are supposed to remain ephemeral and not ever be copied onto a hard drive.
Some users are suspicious of cookies and simply turn off the feature.

Please see Section 8.4 for a further discussion of cookies.

16.3.4 Using Cryptography to Strengthen Hidden Fields, Compound URLs, and Cookies

Many of the problems discussed in this section can be solved by using cryptography to protect the information in hidden fields, compound URLs, and cookies. Cryptography can:

Prevent users from understanding the information stored on their computer.
Allow web server applications to detect unauthorized or accidental changes to this information.

Here are the three examples from the previous sections, recoded to use cryptography.

Username and password authentication:

<INPUT TYPE="hidden" NAME="auth"  VALUE="p6e6J6FwQOk0tqLFTFYq5EXR03GQ1wYWG0ZsVnk09yv7ItIHG17ymls4UM%2F1bwHygRhp7ECawzUm %0AKl3Q%2BKRYhlmGILFtbde8%0A:">

A secure shopping cart:

<INPUT TYPE="hidden" NAME="cart"  VALUE="fLkrNxpQ9GKv9%2FrAvnLhuLnNDAV50KhNPjPhqG6fMJoJ5kCQ5u1gh0ij8JBqphBxdGVNOdja41XJ %0APLsT%2Bt1kydWN4Q%2BO9pW0yR9eIPLrzaDsZxauNPEe7cymPmXwd%2B6c1L49uTwdNTKoS0XAThDzow%3 D%3D%0A:">

A compound URL:

http://www.vineyard.net/cgi-bin/password_ tester?p6e6J6FwQOk0tqLFTFYq5EXR03GQ1wYWG0ZsVnk09yv7ItIHG17ymls4UM%2F1bwHygRhp7ECawzUm %0AKl3Q%2BKRYhlmGILFtbde8%0A:

In each of these cases, the individual human-readable variables have been replaced with a cryptographic block of information. This block is created with the following procedure:^[6]

^[6] This implementation is not designed to protect against the cookie shifting attack. For more information, see slide #29 of http://cookies.ks.mit.edu/pubs/webauth:sec10-slides.pdf.

Take the individual variables that need to be preserved and encode them as a string. (This is called marshalling.)
Prepend a 4-byte timestamp to these variables. (The timestamp protects against replay attacks.)
Compress the data. (This saves space.)
Prepend to the data the length of the string. (Required for decryption with block cipher.)
Encrypt the string using a symmetric encryption function with a secret key.
Calculate an HMAC function of this encrypted string and prepend it to the encrypted string. (HMAC protects all encrypted, compressed, and marshalled data.)
Encode the resulting string with Base64, then escape the non-URL characters and return the resulting string.
Use this escaped, Base64-encoded, encrypted, compressed string for hidden fields, compound URLs, and cookies.

To decode and validate this encrypted string, simply follow these steps in reverse:

Isolate the escaped, Base64-encoded, encrypted, compressed string from the hidden field, compound URL, or cookie.
Unescape the Base64 representation.
Remove the Base64 coding.
Verify the HMAC. If it doesn't verify, then the string has been tampered with. Report an error and return.
Unencrypt the data.
Recover the length and use this to truncate the unencrypted data to the original length. (This step is needed because block encryption functions will append null bytes to data to pad it out to an even block.)
Decompress the compressed data.
Recover the timestamp from the beginning of the uncompressed data. If the timestamp is too old, disregard.
Return the remaining data to the caller, which will decode all of the original variables from the string.

This must all look tremendously complicated and computationally intensive. In fact, it is quite easy to code up and can run very quickly, as MD5 and symmetric encryption functions are quite fast. When we ran the code in Example 16-2 on a very slow 233Mhz K6 computer, for instance, we were able to do 1000 secure_encode( ) operations in just 90 seconds of CPU time, including the time to start up one copy of the Perl 5.05 interpreter that's roughly 90 msec for each secure_encode( ) operation.

Example 16-2. Secure cookie generation and decoding

# # Program to demonstrate secure_encode and secure_decode, two functions # that securely encode and decode timestamped, encrypted strings.  # # Makes extensive use of Perl libraries use Digest::HMAC_MD5 qw(hmac_md5); use CGI; use Crypt::TripleDES; use MIME::Base64; use Compress::Zlib; use strict; my $des3 = new Crypt::TripleDES; #  # Configuration parameters my $passphrase = "Now is the encryption time"; my $digest_key   = "some nasty key"; my $timeout = 7*24*60*60;  # maximum age of tokens, in seconds (this is one week) # secure_encode: # Takes a string and securely encodes it.  Because we use a block cipher # that will pad out the data to the next block, we need to record the # length of the data. It is put in the first four bytes of the data # before encryption. sub secure_encode {     my $tdata  = pack('I',time) . $_[0];                     # Prepend the time (packed)     my $cdata  = compress($tdata); # Compress     my $lcdata = pack('I',length($cdata)) . $cdata;          # prepend the length     my $edata  = $des3->encrypt3($lcdata,$passphrase);       # encrypt     my $hmac   = hmac_md5($edata,$digest_key);               # compute hmac     my $hedata = $hmac . $edata;     return CGI::escape(encode_base64($hedata));              # return hmac . edata } # # Secure decode. Return undef if decryption fails, -1 if timestamp is out of date # and the value otherwise sub secure_decode {     my $hedata = decode_base64(CGI::unescape($_[0])); # get mac & encrypted data     my $hmac  = substr($hedata,0,16); # hmac from data     my $edata = substr($hedata,16);     # Now verify the HMAC     if( hmac_md5($edata, $digest_key) ne $hmac){         print STDERR "DIGEST doesn't verify. \n";         return undef;     }     my $lcdata = $des3->decrypt3($edata,$passphrase);     my $datalen = unpack('I',substr($lcdata,0,4)); # recover the length     my $cdata   = substr($lcdata,4,$datalen);      # recover the compressed data     my $tdata = uncompress($cdata); # get the uncompressed data     # check the timestamp     my $otime = unpack('I',substr($tdata,0,4));     if($otime + $timeout < time){         print STDERR "timeout\n";         return -1;     }     # Return the data that is after the timestamp     return substr($tdata,4); } my $enc = secure_encode("username=simsong&password=myauth11"); print "encode $enc:\n"; print secure_decode($enc),"\n";

only for RuBoard - do not distribute or recompile