Chapter 2: HTTP Security | Security Technologies for the World Wide Web, Second Edition

HTTP is the main application protocol used on the WWW. In this chapter, we overview and briefly discuss HTTP and its basic security features. More specifically , we introduce HTTP and its mode of operation in Section 2.1, overview HTTP user authentication, authorization, and access control in Section 2.2, address HTTP basic authentication, HTTP digest access authentication, and certificate-based authentication in Sections 2.3 to 2.5, discuss the proper configuration of a Web server (i.e., Apache Web server) in Section 2.6, and draw some conclusions in Section 2.7.

2.1 HTTP

As mentioned in Chapter 1, HTTP is a simple request/response protocol that is used between a client (i.e., browser) and a Web server [1]. ^[1] This basically means that the client requests information and the server provides the requested information using the HTTP. The information, in turn , may be represented by Web pages that are static or dynamically created. In many cases, the pages may be written in a specific format or language, such as HTML or XML. In the future, XML will be the preferred language for the information provided on the WWW (and elsewhere). This is particularly true for the use of XML in conjunction with complementary technologies, such as WSDL, SOAP, and UDDI registries (we have briefly reviewed these technologies in Chapter 1).

If the server provides static Web pages, the situation is comparably simple and the pages can be directly retrieved from the server s document tree. If, however, the server must provide dynamically created Web pages, the pages must be created by a specific program in response to an incoming HTTP request message. Historically, the first solution was to have these programs invoked using the Common Gateway Interface (CGI). Although CGI makes it simple to have a Web server perform a specific operation, such as a database lookup, it is not efficient because it requires that a separate program is started and a corresponding process is initialized for each incoming HTTP request message. There are some alternative technologies that can be used instead of CGI. For example, FastCGI is an open Web server interface that solves the performance problems inherent in CGI. ^[2] Also, many vendors provide proprietary application programming interfaces (APIs) for their Web servers. Examples include the Netscape Server API (NSAPI) from Netscape and the Internet Server API (ISAPI) from Microsoft. Last but not least, there are server-side technologies, such as ASP and JSP. The security implications of these technologies are further addressed in Chapter 11.

If a client wants to retrieve a resource (e.g., a static or dynamically created Web page) from a Web server, it must establish a TCP connection to the corresponding port (e.g., port 80 by default) of the server and send a corresponding HTTP request message to the server. In essence, the HTTP request message includes the following components :

A request method that indicates the purpose of the HTTP request. The most important request method is GET. There are, however, other methods defined in the HTTP/1.1 specification (i.e., OPTIONS, HEAD, POST, PUT, DELETE, TRACE, and CONNECT). You may refer to RFC 2616 [1] for a complete and comprehensive description of these methods .
A reference that indicates the resource to which the method should be applied (e.g., http://www.esecurity.ch/index.html). In theory, such a reference may be given in one of the following forms:

A uniform resource locator (URL) [2, 3]; A uniform resource name (URN) [4]; A uniform resource identifier (URI) [5].

In practice, however, URLs are most widely used. Sometimes they are called URLs and sometimes they are called URIs (e.g., in many IETF protocol specifications).

A string indicating the HTTP protocol version (e.g., HTTP/1.0 for version 1.0 of the HTTP);
A MIME-like message containing request modifiers, client information, and possibly some body content.

Again, the exact format and syntax of HTTP messages (i.e., HTTP request and response messages) is specified in [1]. This RFC document is recommended reading for anybody working in the field.

As a working example, consider the situation in which a user wants to retrieve the home page of eSECURITY Technologies Rolf Oppliger. Therefore, the user simply enters www.esecurity.ch in the browser s address or URL field. The browser, in turn, does the following things on the user s behalf :

It uses the Internet Domain Name System (DNS) to retrieve the IP address of the Web server that hosts www.esecurity.ch.
It uses the client system s IP stack to establish a TCP connection to the Web server. ^[3] Since the user has not specified a port number at first place, the browser assumes that the Web server runs at port 80. Any other port is possible but must be specified in the URL (separated with a colon from the rest of the resource reference).
It composes an HTTP request message and uses the TCP connection to send the message to the server (it assumes the use of HTTP by default).

It is now up to the Web server to process the HTTP request message and to send back a corresponding HTTP response message.

In the example given above, the browser would compose an HTTP request message that may look as follows :

 GET http://www.esecurity.ch HTTP/1.0 Host: www.esecurity.ch  Proxy-Connection: Keep-Alive  User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0) Accept: image/gif, image/jpeg, ...  Accept-Language: en  ...

Note that this HTTP request message only includes a header part (i.e., the body part is empty). The header part, in turn, includes a number of HTTP headers. Each HTTP header provides specific information. For example, the Host header specifies that the requested Web site is www.esecurity.ch and the Proxy-Connection header specifies that the TCP connection that is going to be established between the browser and the proxy server must be kept alive and used to serve subsequent HTTP requests. Furthermore, the User-Agent header specifies what browser on what platform the user employs (i.e., Microsoft s Internet Explorer version 5.5 on a Windows NT version 4.0 system), ^[4] and the Accept and Accept-Language headers specify the MIME types and languages accepted by the browser. In addition, there is an increasingly large number of complementary HTTP headers that may be used in HTTP request and response messages.

In response to the HTTP request message, the Web server could send back an HTTP response message that consists of a status line, including the message s protocol version and a success or error code, followed by a MIME-like message containing some server information, entity meta information, and possibly some body content (separated with an empty line from the header). For example, a typical HTTP response message may look as follows:

 HTTP/1.0 200 OK Date: Mon, 03 Dec 2001 12:10:13 GMT Server: Apache/1.3.20 (Unix) PHP/4.0.6 FrontPage/4.0.4.3 Last-Modified: Wed, 03 May 2000 08:01:16 GMT  ETag:"81dfc1-14e-390fdccc"  Accept-Ranges: bytes Content-Length: 334 Content-Type: text/html Proxy-Connection: Keep-Alive <HTML>    <HEAD>     <TITLE>eSECURITY Technologies Rolf Oppliger</TITLE>   </HEAD>    <FRAMESET COLS="23%,*">     <FRAME src="http://www.esecurity.ch/toc.html">    <FRAME src="http://www.esecurity.ch/esecurity.html         NAME="view_frame">   </FRAMESET>    <NOFRAME>   Your browser does not support frames.   </NOFRAME>  </HTML>

The first line of the HTTP response message includes a status code. In this case, the status code 200 means that the HTTP request message is fine (i.e., OK) and that it is served . The HTTP response headers that follow give information either about the server (e.g., the Server header) or the returned resource (e.g., the Date and Last-Modied headers). In either case, a Content-Type header is required to inform the browser about the type of the provided resource. In the example given above, the content type text/ html indicates a text document written in HTML. Finally, the HTTP response message may also include the requested resource (separated with an empty line from the header part). In this example, it is the index.html file that is found at the root of the requested server s document tree.

Most HTTP communication is initiated by the browser and consists of a request to be applied to a resource on some Web server. In the simplest case, this may be accomplished via a single TCP connection between the client and the server. A more complicated situation occurs when one or more intermediaries are present in the request/response chain. According to [1], there are three common forms of intermediary:

A proxy server (or proxy ) is a forwarding agent, receiving requests for a URL in its absolute form, rewriting all or part of the message, and forwarding the reformatted request toward the server identified by the URL.
A gateway is a receiving agent, acting as a layer above some other server(s) and, if necessary, translating the requests to the underlying server s protocol.
A tunnel acts as a relay point between two connections without changing the messages. Tunnels are used when the communication needs to pass through an intermediary (such as a firewall) even when the intermediary cannot understand the content of the messages.

This distinction is important because certain HTTP communication options may apply only to the connection with the nearest , nontunnel neighbor, only to the end-points of the request/response chain, or to all connections along the chain. Also, any party to the communication that is not acting as a tunnel may employ an internal cache for serving requests. The effect of a cache is that the request/response chain is shortened if one of the participants along the chain has a cached response applicable to that request. Not all responses are usefully cacheable, and some requests may contain modifiers that place special requirements on cache behavior (as explained later). In fact, there are a huge variety of architectures and configurations of caches and proxies currently being experimented with or deployed across the WWW. There are many things that need to be said about proxy servers, gateways, and tunnels. These things will be said in Chapter 3.

^[1] In practice, the term Web server is used interchangeably to refer to the computer on which Web pages reside, and the program on the computer that receives HTTP request messages and sends back resources in corresponding response messages.

^[2] Further information about FastCGI is available at http://www.fastcgi.com.

^[3] HTTP communication usually takes place over TCP connections. The default port is TCP 80, but other ports can be used. This does not preclude HTTP from being implemented on top of any other protocol on the Internet, or on other networks. HTTP only presumes a reliable transport; any protocol that provides such guarantees can be used.

^[4] The term Mozilla is still in use today. Remember that from Chapter 1that this is what Mosaic Communications called its .rst browser.