Virtual Private Networks

Virtual private networks (VPNs) are another feature that firewalls have quickly adopted. Firewalls, by the very nature of their placement in the network design, represent a natural device to provide termination for LAN-to-LAN and in some cases remote-access VPNs. The firewall separates a public network from a private network and provides network administrators with a policy-enforcement point. As with IDSs, vendors include the ability to set up VPNs on a firewall as a common feature for any firewall device available on the market. All open source firewalls also provide accommodations for VPN traffic.

To deal with VPNs, firewalls must take into account a variety of new protocols. For IPsec-based VPNs, the firewalls must provide for the capability of handling the two types of IPsec traffic: Authentication Header (AH) and Encapsulated Security Payload (ESP), although AH is relatively obsolete and rarely used as a VPN transport. Each provides distinct challenges to firewalls to allow an IPsec VPN to be set up either to the firewall itself or through it to a dedicated VPN appliance. Other types of VPNs that many firewalls support include Layer 2 Tunneling Protocol (L2TP) as well as the Point-to-Point Tunneling Protocol (PPTP). Finally, many firewalls now also offer the ability to support an SSL VPN. Each is discussed in the following sections.

IPsec VPNs

IPsec-based VPNs have quickly become one of the most prevalent VPN technologies on the market. The IPsec RFCs call for two protocols: AH and ESP. Both of these protocols can be used in one of two ways: transport mode or tunnel mode.

Authentication Header

The AH protocol is used to provide connectionless integrity and data origin authentication for IP datagrams. In addition, AH provides for protection against replay attacks. AH protects a variety of fields within the IP header, and the modification of those fields by a firewall can result in the destination host discarding the packets because of an invalid AH integrity check. AH does not actually encrypt the data itself, however, and provides for no confidentiality of the data being transmitted. Figure 14-3 shows the general format for an AH header.

Figure 14-3. IPsec Authentication Header

The fields in the AH header are as follows:

• Next Header This 8-bit field identifies the type of payload after the AH.

• Payload Length This 8-bit field specifies the length of the AH. This is calculated as the number of 32-bit words minus 2.

• Security Parameters Index (SPI) This pseudo-random value identifies the security association for this traffic.

• Sequence Number This monotonically increasing counter identifies the sequence of this packet within the connection. This is used to prevent replay attacks.

• Authentication Data This variable-length field contains an integrity check value (ICV). This value is computed over the following:

  - IP fields that are immutable or predictable in value upon arrival at the endpoint

  - The AH header itself

  - The upper-level protocol data in the payload

Encapsulated Security Payload

The ESP protocol seeks to provide protection for the entire IP data packet rather than just parts of the packet as AH does. ESP is typically implemented in one of two fashions. In the transport mode, the ESP header is inserted after the IP header and before the upper-layer protocol data. In the tunnel mode, the ESP header is inserted before the IP header and surrounds the IP packet with a new header and trailer and then optionally encrypts the original IP packet. A new IP header is then added for routing to the destination system.

ESP can provide confidentiality and integrity to the communication and can be used to protect just the transport layer segment (Layer 4 and above) or the entire IP data packet (Layer 3 and above). Figure 14-4 shows the format of the ESP header.

Figure 14-4. IPsec Encapsulated Security Payload Header

The fields in the ESP header are as follows:

• SPI This pseudo-random value identifies the security association for this traffic.

• Sequence Number This monotonically increasing counter identifies the sequence of this packet within the connection. This is used to prevent replay attacks.

• Payload Data This variable-length field contains data that is described by the Next Header field.

• Padding These null bytes are used to ensure that the fields fall on 8-byte boundaries.

• Pad Length This 8-bit field identifies the number of padding bytes immediately preceding it.

• Next Header This 8-bit field identifies the type of payload contained in the Payload Data field.

• Authentication Data This variable-length field contains an ICV of the ESP data but not including the authentication data.

IPsec Modes

IPsec VPNs can operate in one of two modes: transport mode or tunnel mode. IPsec in transport mode can only be used by the device that generates the original data packet and protects only the upper-layer protocols of the IP packet and not the original IP header, as shown in Figure 14-5. In tunnel mode, the entire original IP packet is encapsulated within a new IPsec header and protected. Tunnel mode must be used for VPN devices that encapsulate packets that other devices (such as a computer on a network) generate. This new header includes new IP information outside the IPsec header, as shown in Figure 14-6.

Figure 14-5. IPsec ESP in Transport Mode

Figure 14-6. IPsec ESP in Tunnel Mode

When configuring a firewall to either terminate or to allow IPsec traffic to pass, you must consider the mode. Most firewalls perform some Network Address Translation (NAT) function on the IP packets flowing through them. The modifications, namely the change of the source IP address, made by the NAT process in the firewall can impair the ability of IPsec traffic to travel through the firewall because any changes to the headers of the IP packets may result in the termination point of the VPN determining that the packets were modified en route (and rightly so) and therefore invalid. This is a result of the fact that a component of IPsec, the Internet Key Exchange (IKE) protocol, embeds the source IP address in its payload. When the source address of the packet (which will be the NAT device) does not match the address in the IKE payload, the destination system (the VPN termination point) drops the packet. To work around this issue, many vendors chose to encapsulate the VPN traffic in another packet via a process known as NAT Traversal (NAT-T).

NAT-T typically adds a UDP header (although TCP can be used) that encapsulates the ESP header shown in Figure 14-6. This UDP header allows a NAT device to perform two primary tasks. First, it allows the NAT device to distinguish between IPsec data streams by using UDP ports to identify one stream from another. This allows multiple devices to use IPsec behind the NAT device. Second, NAT-T puts the original source computer IP address into a NAT Original Address (NAT-OA) payload, allowing the receiving computer to access the original source IP address information for the purposes of performing the IKE checksum process. This allows IPsec traffic to traverse NAT devices, hence why it is known as NAT Traversal.

SSL VPNs

SSL VPNs are a simple VPN technology that uses a common web browser as a means of accessing systems and services within a network. The client browses to the SSL VPN device (using TCP port 443, just as when the client connects to a secure website) and authenticates to access the SSL VPN connection. This type of VPN requires no special considerations in terms of the firewall because all traffic occurs over TCP port 443. Therefore, this is the simplest type of VPN that you can deploy behind a firewall and is not affected by the NAT process of the firewall. However, SSL VPNs typically require that the back-end applications and systems support this method of access, which tends to limit the use of SSL VPNs to specialized situations (for example, used in conjunction with Citrix or Microsoft Outlook Web Access).

Few firewalls support SSL VPN functionality, which is a limitation of using firewalls as SSL VPN termination points. In conjunction with the application limitations previously discussed, this lack of support has limited the adoption of SSL VPNs in many environments.