3.6 HTTP Security

Many security mechanisms exist for HTTP, typically designed originally to solve various e-commerce scenarios. HTTP is concerned with only a few major threats.

User Identification

Many Web servers need to identify users. Some sites limit access to resources, and these sites need to identify users to see if the operation is allowed (access control). Some sites store user preference information; MSN and Yahoo! build custom home pages for each logged-in user. Users are typically identified with a username and password. Certificates or public keys could also identify users, but not very many Internet users have these yet.

Server Identification

The most common scenario in which server identity must be securely established is e-commerce. Users send their credit card numbers to sites like Amazon.com to make purchases. It's very important that the server be truly Amazon.com and not a site run by an attacker impersonating a known site in order to gather and use credit card numbers. Servers are usually identified with certificates signed by trusted certificate authorities such as VeriSign.

Confidentiality

Information sent using HTTP should not always be publicly readable. In e-commerce, this includes messages containing credit card numbers from clients to servers and messages traveling from servers to clients and that contain order confirmations with home delivery addresses. In authoring/publishing scenarios, this information includes any message containing a resource body that isn't supposed to be publicly readable. Any information sent over TCP without encryption must be considered publicly readable, because it's quite easy to sniff network packets at some point between the client and the server.

Most of these threats can be addressed with a good transport security mechanism. Confidentiality is ensured with transport channel encryption (SSL/TLS). Message integrity, or tamper prevention, is ensured via signatures on transport layer messages. Server identification is handled via certficates exchanged when the transport channel is established.

The remaining threat is user identification, which is addressed with HTTP authentication within the HTTP message. There are several HTTP authentication mechanisms and a framework to allow clients and servers to choose one. Authentication works even better when combined with a transport security mechanism, because the transport security provides extra protection for confidentiality and message integrity, both of which help prevent impersonation attacks.

Transport security and HTTP authentication are both easy to provide. An HTTP or WebDAV implementer (or somebody designing a custom HTTP or WebDAV application) is unlikely to have to reimplement any of these security mechanisms.

3.6.1 Basic Authentication

The Basic authentication mechanism and the framework for choosing an authentication mechanism in HTTP are both defined in an RFC [RFC2617]. Basic authentication only addresses user identification, and it does so very poorly.

The Basic authentication challenge includes the Basic identifier string and optionally the realm or domain the user should log in to.

 WWW-Authenticate: Basic realm="example.com" 

The realm is provided so that the client can prompt the user to log in to the site. The client might connect to the site with an IP address and have no other way to identify the site. The client displays the realm string to help the user recall the username and password to enter to log in to that site. The client may also cache usernames and passwords along with the realm so that the user isn't continually prompted for the same password. The realm should be a domain name registered with the site host so that it doesn't overlap with other sites.

The client authenticates by sending the Authorization header on the request. The username and password are concatenated together, transformed using base-64 encoding, and included in the header value.

 Authorization: BASIC bGlzYTp0ZXN0 

The Basic Authorization header value is always the same for a given username and password. That means that it's trivial for a hacker to reuse the header value on new HTTP requests, impersonating another user and gaining access to his or her resources. Even worse, it's trivial to capture TCP packets, unencode this header value, and recover the original username and password.

Since Basic authentication exposes the user's name and password over the wire, it isn't recommended for use over unprotected transport channels. Basic authentication is acceptable if it's used on messages that are encrypted within an SSL/TLS connection, using the server's public key to make the channel confidential.

What about sites that don't care about security? Some sites, like newspapers, only want the user to log in so that the site can track usage patterns or demographic information. The site administrators don't care if one user impersonates another, because their statistics don't have to be perfectly accurate. The site administrators don't care about access control, because all the articles are actually publicly readable. However, this setup does a great disservice to the site's users.

Since many users choose the same passwords for several sites, an attacker who has captured one Basic Authorization header can often use that to break into an account on a more secure site. Users have a hard time remembering many passwords, and so they reuse them frequently. Users also have a hard time distinguishing which connections are secure and which aren't, especially if sites change their policy. For that reason, any site using Basic authentication over a clear channel is exposing passwords that may not be unique and that may allow other sites to be attacked. This hurts users whose accounts are broken into as well as the other sites.

3.6.2 Digest Authentication

Digest authentication is the general recommendation for unencrypted HTTP authentication. Most Web servers and clients now support Digest, although there have been reported interoperability problems between MS IE and Apache and between Mozilla and MS IIS. Navigator is the only browser that does not support Digest in any recent version.

Digest addresses user identity and does a much better job of protecting against user impersonation. A Digest Authentication header value may not be reusable by an attacker. It uses a nonreversible digest algorithm (usually MD5 [RFC1321]), so an attacker cannot extract the username and password, even if the entire transaction is captured.

Digest authentication protects against replay attacks by using different seed or nonce values for each request. The header can't be copied and reused on a new HTTP request if the nonce changes.

Digest addresses the message-tampering threat as well, protecting against certain man-in-the-middle attacks. An attacker could capture the client's HTTP request with Digest authentication and modify the HTTP request before forwarding it to the server. This could be quite harmful in e-commerce if an attacker could replace an order for one gold coin with an order for 100 or have the delivery sent to another location. Message integrity protection prevents tampering with the message body. The client calculates a checksum for the original message body and sends it in the Authorization header for the server to compare. It's quite difficult to generate a bogus message body that has the same checksum as the original message. Message integrity does not provide confidentiality, however Digest authentication does not involve any message body encryption. The headers and their values are also not protected by Digest.

As with Basic, Digest authentication is advertised in the WWW-Authenticate header when the server challenges the client to authenticate. The Digest challenge includes quite a bit of challenge information.

 WWW-Authenticate: Digest realm="www.example.com",    stale=false, nonce="ec2cc00f21f71acd35ab9be057970609",    qop="auth", algorithm="MD5" 

The client needs to use all the information in this challenge to correctly authenticate. Here is what each parameter means:

• As with Basic, the realm is used to prompt the user to log in to the correct server. The client may temporarily cache usernames and passwords by associating them with the realm.

• The algorithm tells the client how to compute the digest and the checksum. The MD5 algorithm is the only one defined.

• The nonce is a unique authentication challenge: The client must use the nonce when computing the digest. If the nonce changes in every authentication challenge, then the user is protected from replay attacks. When the nonce changes, the Authentication header value changes too, and a previously valid header value is no longer valid.

• The qop (quality of protection) value indicates that the server is merely looking for the client to authenticate. Another possibility is for the server to ask for message integrity as well as authentication (qop=auth-int).

• The stale directive advises the client on whether it's appropriate to prompt the user for the password again or use a cached value for the password. The server would send stale=true when it needs the client to rechallenge the user.

WebDAV clients and servers must support Digest authentication. This really shouldn't be too hard, because many programming languages now have HTTP libraries supporting Digest authentication, and many Web servers can be extended while still using their Digest implementation.

• The Java 2 Platform now includes a Digest authentication implementation.

• Two modules for Python (httpx and urllib2) support Digest authentication.

• At least one library for C supports Digest (libwww and neon for clients).

• WebDAV libraries for various programming languages should also support Digest.

• Custom servers extending Apache can use the mod_auth_digest module.

• Custom servers extending IIS 5.0 can use the IIS 5.0 built-in Digest implementation.

Clients should support both Basic and Digest authentication, but clients should never choose Basic authentication if Digest is offered by the server. Many servers advertise both in order to support all kinds of unknown clients, so it's up to the client to choose the more secure mechanism. Even if the TCP connection is encrypted, that might only be true for the first hop, and information may be sent in the clear after that hop, exposing the user's password.

NTLM Authentication

The HTTP authentication framework is extensible. The server can extend the standard WWW-Authenticate header to advertise custom mechanisms, and the client can use the standard Authorization header to send authentication information other than Basic or Digest.

NTLM authentication is the Microsoft proprietary authentication solution and the most common nonstandard authentication mechanism. IIS 5.0 advertises support for NTLM authentication (perhaps along with Basic and Digest) when it challenges the client to authenticate. The NTLM mechanism is undocumented; however, it has been analyzed by independent experts. It is a challenge-response mechanism, more secure than Basic, but it seems to be less secure than Digest.

3.6.3 Transport Security

Many Internet application protocols use TCP as a transport layer by making a TCP connection and sending all data in TCP packets. There ought to be a way to provide confidentiality and server authentication for the transport layer so that all these application protocols can reuse the same secure connection technology. The only transport layer security mechanisms worth mentioning, used by practically all modern application protocols, are the SSL/TLS series of protocols.

Transport security typically solves several problems at the same time. These are the most common functions provided. Some of these functions can be turned off, but very frequently they all are used.

1. It authenticates the server through a certificate that includes the server's domain name and is signed by a trusted certificate authority. This is quite useful in e-commerce where users are loathe to send their credit cards to any old Web server.

2. It encrypts the entire communication channel so that all data on the channel is private.

3. It signs each message so that the recipient can make sure the message was not tampered with in transit.

Note that transport security doesn't address user identification very often, although it can in theory. With HTTP, it's often easiest for the administrator to configure HTTP authentication rather than configure transport layer client authentication.

The IETF recommends the Transport Layer Security (TLS) protocol [RFC2246]. TLS was standardized in January 1999 but is not as new as it appears, because it is a careful refinement of Secure Socket Layer 3 (SSL3). SSL3 itself is based on SSL2, which was published in 1995 [Hickman95]. SSL3 is the most widely implemented version as of 2002, but TLS is slowly replacing SSL3 [Rescorla01]. Since SSL/TLS handles upgrade cases, the application protocol rarely needs to worry about versions.

WebDAV client and server implementors should choose a SSL/TLS toolkit so that transport security doesn't have to be reimplemented. Open-source security toolkits are a great resource for impementors because with open source any independent expert can examine the source code to discover potential coding errors and security holes (and security experts aren't shy about reporting these). The most frequently used open-source toolkits are OpenSSL for C and PureTLS for Java. If a HTTP implementation plugs in to a toolkit correctly, then the best quality mechanism available to both client and server will be chosen automatically for each connection.

Newest and Best? Go with Tried and True

Security mechanisms and implementations should be reused wherever possible. A good security mechanism like TLS takes years of experience and refinements by experts. A new mechanism, even if it uses the newest, greatest cryptography, is likely to have all kinds of undiscovered problems. A good security implementation takes years of careful development work by rare and highly paid experts. The developer must pay careful attention to every possible threat and every coding error that could open a theoretically secure protocol to attack. A good security implementation may be in beta a long time as implementors slowly become more certain that holes have been plugged and changes do not open new holes.

WebDAV servers supporting SSL/TLS should also support the CONNECT method (described in Section 3.3.10). SSL and TLS: Designing and Building Secure Systems [Rescorla01] has detailed information on HTTP and SSL/TLS interactions, including CONNECT, proxy interactions, and various threats.

All WebDAV clients should support SSL and TLS because distributed authoring frequently requires much better authorization and confidentiality than regular Web browsing does. Some WebDAV repositories only support SSL or TLS connections, rejecting ordinary TCP connections altogether, so a client that doesn't support SSL/TLS can't interoperate with these repositories at all.