14.4 IPSEC PROTOCOLS

Team-Fly

According to the terminology introduced in the OSI security architecture, the IPsec protocols provide the following security services:

A data origin authentication service;
A connectionless data integrity service (including protection against replay attacks);
A data confidentiality service;
An access control service;
A limited traffic flow confidentiality service.

The security services are provided at the Internet layer, offering protection for IP and upper-layer protocols. As mentioned previously, the security services are provided by two subprotocols, namely, the AH and the ESP. Each protocol can be used to protect either only the upper-layer payload of an IP packet or the entire IP packet. This distinction is handled by considering two different modes of operation:

Transport mode is used to protect the upper-layer payload of an IP packet.
Tunnel mode is used to protect an entire IP packet (in this case, IP encapsulation is used as an enabling technique).

Figure 14.3 illustrates the IPsec transport and tunnel modes. In transport mode, an IPsec header (i.e., an AH or ESP header) is inserted between the original IP header and payload (i.e., the TCP segment or UDP datagram). This is true for both IPv4 and IPv6:

In IPv4, the transport mode IPsec header appears immediately after the IP header (with or without options), but before the upper-layer protocol data (e.g., TCP or UDP).
In IPv6, the IPsec header appears after the base IP and some extension headers, but before destination options and the upper-layer protocol data.

click to expand
Figure 14.3: IPsec transport and tunnel modes.

In tunnel mode, the original IP packet is encapsulated into another IP packet. What this means is that there are two IP headers:

An inner IP header that carries the original IP header (specifying the original source and destination IP addresses);
An outer IP header that carries the new IP header (specifying new source and destination IP addresses).

The tunnel mode IPsec header appears between the outer IP header and the inner IP header.

Both IPsec protocols—AH and ESP—can operate in either transport or tunnel mode. Transport mode is typically used to secure IP traffic between two endpoints (i.e., computer systems), whereas tunnel mode is typically used to secure IP traffic between two points that are not necessarily the endpoints of the communications. For example, one of the points may be a security gateway for a corporate intranet. In this case, the IP traffic is encapsulated (i.e., using IPsec in tunnel mode) between the remote system and the security gateway (making sure that the systems located on the corporate intranet must not be able to handle IPsec). Note that whenever either endpoint is a security gateway, IPsec must be used in tunnel mode (in the case where traffic is destined for a security gateway, e.g., SNMP commands, the security gateway is acting as a host and transport mode is also allowed).

14.4.1 Authentication Header

The IPsec AH protocol provides data origin authentication and connectionless data integrity for IP packets (collectively referred to as "authentication" in this section). The precision of the authentication is a function of the granularity of the SA with which AH is employed. Depending on which cryptographic algorithm is used and how keying is performed, the AH may also provide non-repudiation of origin services. Finally, the AH may offer an antireplay service at the discretion of the receiver, to help counter specific denial-of-service attacks.

The IANA has assigned the protocol number 51 for the AH protocol, so the header immediately preceding the AH must include 51 in its protocol or next header field. As specified in RFC 2402 [15] and illustrated in Figure 14.4, the AH header consists of the following fields:

An 8-bit Next Header field;
An 8-bit Payload Length field;
A 16-bit field that is reserved for future use;
A 32-bit SPI field;
A 32-bit Sequence Number field;
A variable-length n*32-bit Authentication Data field.

click to expand
Figure 14.4: he authentication header (AH) format.

The authentication data is computed by using an authentication algorithm and a cryptographic key specified in the corresponding SA. The sender computes the data before sending the IP packet, and the receiver verifies it upon reception. Several algorithms for authentication data computation and verification have been proposed in the past. In particular, RFC 2403 [16] suggests the use of the HMAC construction with MD5 and RFC 2404 [17] suggests the use of the same construction with SHA-1 (instead of MD5). The HMAC construction is fully explained in [27]. In short, the HMAC construction takes as input the message M and the authentication key K, and produces as output the following expression:

This expression looks more complicated than it really is. To compute HMA_CK(M), the key K and an inner pad value ipad (ipad refers to the byte ox36 repeated several times) are first added modulo 2. The result is then concatenated with the message M and hashed with the one-way hash function h (which can be either MD5 or SHA-1). Similarly, the result is then concatenated with the sum of K and an outer pad value opad (opad refers to the byte Ox5C repeated several times) modulo 2. Finally, this result is hashed with the appropriate one-way hash function h (MD5 or SHA-1), and the resulting authentication data is truncated to 96 bits.^[10] Depending on the one-way hash function in use, the resulting HMAC constructions are called HMAC-MD5-96 (in the case of MD5) and HMAC-SHA-1-96 (in the case of SHA-1).

As the AH protocol does not provide data confidentiality services, implementations thereof may be widely deployed, even in countries where controls on encryption would preclude deployment of technology that potentially offered data confidentiality services. Consequently, AH is an appropriate protocol to employ when confidentiality is not required.

14.4.2 Encapsulating Security Payload

As its name suggests, the IPsec ESP protocol uses IP encapsulation to provide data confidentiality and partial traffic flow confidentiality (in tunnel mode and with the invocation of padding data to hide the size of an IP packet). Similar to the AH, the ESP protocol may also provide authentication (referring to data origin authentication and connectionless data integrity services). Note, however, that the scope of the authentication offered by the ESP is narrower than that for the AH (i.e., the IP headers below the ESP header are not protected). If only the upper-layer protocols need to be authenticated, then ESP authentication is an appropriate choice and is more space efficient than use of an AH encapsulating an ESP.

The IANA has assigned the protocol number 50 for the ESP protocol, so the header immediately preceding the ESP must include 50 in its protocol or next header field. The ESP format is specified in RFC 2406 [19] and illustrated in Figure 14.5. It consists of the following fields:

A 32-bit SPI field (not encrypted);
A 32-bit Sequence Number field (not encrypted);
A variable-length Payload Data field;
A variable-length Padding field;
An 8-bit Pad Length field;
An 8-bit Next Header field;
In addition, the ESP may also include a variable-length n*32-bit Authentication Data field.

click to expand
Figure 14.5: The encapsulating security payload (ESP) format.

In RFC 2405 [18], the DES in cipher block chaining (CBC) mode with an explicit initialization vector (IV) is introduced as the default algorithm to encrypt the ESP Payload Data field. But this default algorithm may be replaced by any other algorithm at will. For example, RFC 1851 specifies the experimental use of 3DES. In the future, it is possible and very likely that we will see more AES implementations instead of DES or 3DES implementations. Unfortunately, export, import, and use of specific encryption algorithms may be regulated in some countries. The algorithms for computing the authentication data are the same as the ones suggested for the AH.

Note that both AH and ESP are also vehicles for access control, based on the distribution of cryptographic keys and the management of traffic flows relative to these security protocols. Also note that full protection from traffic analysis is not provided by any of the two IPsec subprotocols. At the most, tunnel mode ESP can provide a partial traffic flow confidentiality service. In fact, the ESP protocol can be used to create a secure tunnel between two security gateways. In this case, anyone eavesdropping on the communications between the security gateways is not able to see what hosts are actually sending and receiving IP packets from behind the security gateways. Nevertheless, it is fair to mention that only a few Internet users worry about traffic analysis at all.

A final word should be said about the interdependence between data compression and encryption. Note that when encryption is employed at the Internet layer, it prevents effective data compression by lower protocol layers. Unfortunately, the suite of IPsec protocols in its current form does not provide support for a data compression service. Such a service may be provided by higher layer protocols, or, in the future, by IP itself.

^[10]The truncation was introduced because of a desire to achieve a specific packet alignment goal, to avoid devoting all 128 or 160 bits to the authentication function, and to have a uniform size MAC whether MD5 or SHA-1 is employed.

Team-Fly