2.4 Digest access authentication

Due to the fact that the HTTP basic authentication scheme must be considered to be weak and vulnerable, the complementary and inherently more secure HTTP digest access authentication scheme has been specified in [7] and submitted to the Internet standards track. Note, however, that the HTTP digest access authentication scheme still suffers from many known limitations and weaknesses (as discussed at the end of this section), and that it is intended as a simple replacement for the HTTP basic authentication scheme. More secure HTTP authentication schemes can be designed using public key certificates or authentication and key distribution systems, such as Kerberos [9].

Unfortunately (and contrary to the HTTP basic authentication scheme), the HTTP digest access authentication scheme is not widely deployed and supported by Web browsers and servers. ^[8] On the client side, for example, the scheme is supported by the latest releases of Microsoft s Internet Explorer and Opera (the scheme is not supported by Netscape Navigator). On the server side, the scheme is supported by Microsoft IIS and Apache. ^[9]

Like the HTTP basic authentication scheme, the HTTP digest access authentication scheme implements a simple challenge-response mechanism to verify that the user knows a secret he or she shares with the server (i.e., the password). Unlike the HTTP basic authentication scheme, however, the verification is done without actually sending the secret in the clear. Instead, the HTTP digest access authentication scheme employs a one-way hash function (typically MD5 ^[10] ) to compute (on the browser side) and verify (on the server side) a digest value that is used to proof knowledge of the secret.

More specifically , when a browser requests a resource that is protected using the HTTP digest access authentication scheme, the server challenges the browser using a nonce (i.e., a randomly chosen value). The browser, in turn , must respond with a valid digest value to authenticate itself to the server. The digest value is computed from the following input parameters:

The username;
The user password (or a hash value thereof);
The nonce;
The HTTP access method;
The URL of the requested resource.

Roughly speaking, the digest value is computed as follows :

h(h(A1):nonce:h(A2))

In this formula, h represents the one-way hash function, ^[11] A 1 the expression that consists of the username, the realm string, and the user password (each component separated with a colon ), and A 2 the expression that consists of the HTTP access method and the requested URL (again, separated with a colon). There are some options that complicate the formula, but in principle, this is the way the browser has to compute the digest value it must encode as a response. Note that the user password is not sent in the clear. Instead, it is used as a secret input to the one-way hash function. As with the HTTP basic authentication scheme, the usernames and passwords must be prearranged and distributed out-of- band .

Let us assume that a browser wants to retrieve the index.html file located in the protected directory /Demo/HTTPDigestAccessAuthentication/ at www.esecurity.ch. Again, the browser does not know that this file is protected using the HTTP digest access authentication scheme. So it sends sends out a normal-looking HTTP request message. Remember from our previous discussions that this message may start with the following request line:

GET http://www.esecurity.ch/Demo/HTTPDigestAccessAuthentication/HTTP/1.0

All other HTTP request headers remain the same as in the previous example(s). After having received the HTTP request message, the Web server recognizes that the requested file is located in a directory of the Web server s document tree that is protected using the HTTP digest access authentication scheme. So it returns an HTTP response message that includes the following two characteristic lines (among other lines and HTTP headers):

 HTTP/1.0 401 Unauthorized ... WWW-Authenticate: Digest   realm="HTTP Digest Access Authentication Demo",   nonce="1011598310"

Again, if the server were an HTTP proxy server, it would return an HTTP response message with a 407 status code and a Proxy-Authenticate header. This case is not discussed in this book.

Contrary to the HTTP basic authentication scheme, the WWW-Authenticate header includes the keyword Digest (referring to the HTTP digest access authentication scheme instead of Basic referring to the HTTP basic authentication scheme) as well as a comma-separated list of parameters. In our example, there are only two parameters (i.e., realm and nonce ) with corresponding values:

The realm parameter carries a string that is displyed to the user so he or she knows which username and password to use. As illustrated in Figure 2.3 for Opera, most browsers display this string in the prompt in which they ask the users to type in their username and password.

Figure 2.3: The Opera 6.0 ˜Password required prompt using the HTTP digest access authentication scheme. ( 2002 Opera Software.)
The nonce parameter carries the random value that is used by the server to challenge the user (or the browser, repectively). The value is uniquely generated each time a 401 response message is compiled. The contents of the nonces are implementation dependent. An exemplary procedure to generate nonces (using time stamps and a secret key that is known only to the server) is given in [7]. In our example, the nonce is 1011598310 .

As further addressed in [7], there are many other parameters that can be used in the WWW-Authenticate header. Most of them are optional. To keep the discussion sufficiently simple, we do not look into the syntax and semantics of these parameters.

In response to the HTTP response message, the browser prompts the user to enter his or her username and password. From a graphical user interface s point of view, this is very similar to the way the user is prompted in the HTTP basic authentication scheme. Figure 2.3 illustrates the prompt used by the Opera browser. If the user entered the requested information (i.e., the username and password), the browser would compute a response value and return it as part of an Authorization header in a second HTTP request message to the server. In our example, the Authorization header would look as follows:

 Authorization: Digest   username="rolf",   realm="HTTP Digest Access Authentication Demo",   uri="http://www.esecurity.ch/Demo/HTTPDigestAccessAuthentication/",   algorithm=MD5,   nonce="1011598310",   response="2cbdf234349bbfbfa8460c2410acb445"

In this header, the response parameter carries the digest value that is needed to authenticate to the server. It is a 32-bit hexademical value that carries a one-way hash value (i.e., an MD5 hash value in this example) that is computed as described above. Due to the fact that the WWW-Authenticate header may include many optional parameters and that implementations should be compatible with a previous version of the HTTP digest access authentication scheme, ^[12] the actual format and procedures to compute and verify the hash values is quite complex. Consequently, we have used a simplified formula and you may refer to [7] to get a more comprehensive description and specification, as well as examples of HTTP request and response messages that conform to this specification.

Upon receiving an HTTP request message with a proper WWW-Authenticate header, the server must check the validity of the response value. In particular, it must look up the A 1 hash value that corresponds to the submitted username, recompute the digest value, and compare the result to the response that was provided by the browser. If the values match, the user is assumed to be authentic . Note, however, that the server does not need to know the user password in the clear. It is sufficient for the server to know the A 1 hash value (i.e., the hash value of the username, the realm string, and the user password).

In the following chapter, we address the implications of proxy servers and firewalls for Web applications. As with the HTTP basic authentication, the use of proxy servers must be completely transparent in HTTP digest access authentication. That is, the proxy servers must forward the relevant headers (i.e., the WWW-Authenticate, Authorization , and some other headers) as they are. If a proxy server wanted to authenticate a client before a request is forwarded to a Web server, it would have to use appropriate Proxy-Authenticate and Proxy-Authorization headers as specified in [1]. Consequently, Web server authentication and HTTP proxy server authentication may coexist in the same challenge/ response messages.

There is a potential difficulty in using an HTTP authentication scheme (i.e., HTTP basic authentication or HTTP digest access authentication) together with caching mechanisms implemented by HTTP proxy servers. Note that one goal of a proxy server is to cache resources that have been downloaded once to serve requests that are issued by multiple browsers. Consequently, if a resource has been downloaded by an authenticated browser, the resource may end up in a proxy cache, from where it may be redistributed to multiple (not authenticated) browsers. To protect against this redistribution, HTTP (since version 1.1) specifies that when a proxy server has received an HTTP request message containing an Authorization header, and a response message from relaying that request, it must not return that response message as a reply to any other request, unless one of the following two cache-control directives was sent in the corresponding original HTTP response message:

If the original HTTP response message includes the must-revalidate cache-control directive, the proxy server can cache the resource and use it to serve further requests for the same resource. Each time the resource is requested, however, the proxy server must first reauthenticate the browser (using the HTTP request headers from the new request to allow the origin Web server to authenticate the browser).
Alternatively, if the original HTTP response includes the public cache-control directive, the proxy server can cache the resource and use it to serve further requests for the same resource (without browser reauthentication).

In summary, the HTTP digest access authentication scheme solves the most severe security problem of the HTTP basic authentication scheme, namely, that passwords are transmitted in the clear (or in Base-64-encoded form that is equivalent to the unencrypted form). Instead of sending the username and password to the server, the browser uses the password to properly compute a response for the challenge provided by the server. As such, the password is never transmitted across the Internet. Provided that the user has picked a good (i.e., hard to guess) password, it is computationally infeasible for an attacker to derive the password from the response. For further protection, the user password may not be stored in the clear on the server side (where it could be stolen by someone with access to the server). Instead, only the hash value of the password may be stored. This is similar to the way that contemporary operating systems, such as UNIX or Windows NT, store passwords. As a final precaution, the requested URL is part of the response. Consequently, if the response is intercepted by an eavesdropper who attempts to play it back to gain illegitimate access to resources, he or she will be able to get access only to that single URL. More specifically, he or she will be unable to generate new responses to gain access to resources that are found in other branches of the document tree. Servers can further protect themselves against replay attacks by adding a timestamp to the nonces so that responses automatically expire after a relatively short period of time.

The HTTP digest access authentication scheme is intended as a simple replacement for the HTTP basic authentication scheme, and nothing more. In spite of all of its security features, the HTTP digest access authentication scheme still suffers from known limitations and weaknesses. For example, both the HTTP basic authentication scheme and the HTTP digest access authentication scheme are vulnerable to the man-in-the-middle attack ( mainly because the server does not authenticate itself to the browser before it sends out the challenge). ^[13] Also, the HTTP digest access authentication scheme is (still) a password-based system and suffers from all the problems of such a system.

For example, digest authentication requires that the authenticating party (usually the server) store some data derived from the username and password in a user password file associated with a given realm. The security implications of this are that if this file is compromised, an attacker gains immediate access to documents on the server using this realm. On the other hand, a brute force attack would be necessary to obtain a user s password.

This is why the realm is part of the hashed data stored in the file. It means that if one digest authentication password file is compromised, it does not automatically compromise others with the same username and password (though it does expose them to brute force attack). This is somewhat similar to the UNIX salt mechanism. There are two important security consequences of this:

The user password file must be protected as if it contained unencrypted passwords (that is why it is usually not stored in the document tree).
The realm name should be unique among all realms that any single user is likely to use. In particular, a realm name should include the name of the host doing the authentication (contrary to the example given previously in this chapter).

Furthermore, no provision is made in the specification of the HTTP digest access authentication scheme for the initial arrangement between the user and server to establish the user password. Consequently, the HTTP digest authentication scheme does not provide a complete answer to the need for security on the WWW. Also note that the HTTP digest access authentication scheme is only an authentication scheme that does not provide any data confidentiality or integrity services. This is where cryptographic security protocols, such as the Secure Sockets Layer (SSL) and Transport Layer Security (TLS) protocols, come into play.

^[8] The major reason for this astonishing fact is that those products typically implement the SSL and TLS protocols. If the browser and server are communicating with HTTP on top of SSL or TLS (using HTTPS), the problem of password sniffing automatically goes away. The encrypted channel is set up before any HTTP header passes across the network, so the username and password are part of the encrypted SSL data stream and cannot be sniffed accordingly .

^[9] Digest access authentication as specified in [7] is implemented by the module mod_auth_digest. There is an older module, mod_digest, which implemented the older digest authentication scheme specified in RFC 2069.

^[10] An optional header may allow the server to specify the algorithm that must be used to create the one-way hash value. By default the MD5 algorithm as specified in RFC 1321 [10] is used. As further explained in Chapter 4, an MD5 hash value is 128 bits long. As such, it can be represented in 32 ASCII printable characters that each represent a hexadecimal number.

^[11] Refer to Section 4.2 for an introduction to one-way hash functions.

^[12] As mentioned before, this version is called ˜ ˜HTTP Digest Authentication and it is specified in RFC document 2069.

^[13] In a man-in-the-middle attack, an attacker spoofs a server and requests a browser to provide a user password.