In Chapter 7, we discussed many of the characteristics of IPSec. Before we examine how IPSec can be implemented optimally, let's review the three types of IPSec architectures:
What's the difference between using SSH or SSL and using IPSec? Although SSH, SSL, and IPSec might provide similar encryption capabilities, they are different from a functional standpoint. For every additional application you use through SSH or SSL, you have to establish additional connections and tunnels. IPSec makes one connection from a client to the remote VPN gateway or host, and it tunnels all application traffic through that one connection. Certain IPSec architectures can also conceal IP addresses, which is a significant security consideration in some environments. From a perimeter defense perspective and from a usability perspective, implementing VPNs using IPSec instead of SSH or SSL has many advantages. Let's look at how IPSec-based VPNs fit into perimeter defenses.
IPSec Client Integration
Most operating systems in use today include native IPSec clients, although some still require a separate IPSec client program to be acquired, installed, and configured. Some organizations also choose to use IPSec clients other than those built in to their systems; this is most often done to take advantage of additional features offered by the clients or to achieve full interoperability with a certain IPSec gateway (that is, using the same vendor for both the IPSec clients and IPSec gateway). Besides the additional time and resources needed to deploy third-party IPSec clients, such software also modifies the operating system's networking functions, which can cause operational problems.
On Windows systems, most nonnative IPSec clients fall into one of two categories:
For UNIX-based IPSec clients, multiple implementation methods are available. IPSec support can be added directly to the kernel, added as a new device driver that is recompiled into the kernel, or added as a loadable kernel module. Examples of free UNIX-based IPSec clients include Openswan (http://www.openswan.org/) and strongSwan (http://www.strongswan.org/).
As already mentioned, IPSec clients sometimes require routing changes to be made on the host. This is particularly true when a client needs to contact internal hosts and external hosts; only the traffic destined for the internal hosts must be handled using IPSec, although all the traffic could be. This isn't just because of the way the client software works; often it is due to the organization's security policy. It might be desirable to route all traffic through the VPN connection and then permit the remote VPN gateway to route traffic to external or internal hosts as appropriate. Of course, this causes a significant performance hit as compared to allowing the VPN client to make direct requests to other external hosts without utilizing the VPN connection to do so.
There is one other important point to know regarding IPSec client software. If you are making a connection between a single external host and a VPN gateway, you should configure the client software so that other hosts cannot use the tunnel. If you want to connect a remote network to your VPN gateway, you have to configure the IPSec software on the client side as a gateway, of course, to pass through traffic only from the authorized hosts on that local network.
IPSec Server Integration
IPSec servers can be deployed to different types of devices. Chapter 12, "Fundamentals of Secure Perimeter Design," contains a good discussion of VPN basics. To quickly review, the three most commonly used systems are as follows:
Although most devices implement IPSec according to standards, some have proprietary IPSec implementations that deviate from standards. If a VPN server runs a proprietary IPSec implementation, its users might be required to use a particular IPSec client, particularly in order to take advantage of proprietary features.
IPSec Perimeter Defense Adjustments
IPSec requires some interesting changes to perimeter defenses. Encapsulating Security Payload (ESP) mode uses IP protocol 50, whereas Authentication Header (AH) mode uses IP protocol 51. The Internet Key Exchange (IKE) negotiation uses UDP port 500. However, IPSec implementation can result in other things as well. NAT is often incompatible with IPSec. Because AH mode makes authentication value calculations based on the entire packet, which includes the source and destination IP addresses, any NAT must occur before IPSec is used on the packets. ESP mode does not have the same problem because it does not include the entire header when it makes its authentication value calculations. In general, if you want to use NAT, you should use ESP to provide authentication instead of ESP. However, there are still cases where ESP and NAT do not work well together, such as when a NAT mapping times out, causing the port used by IKE to change.
This becomes more complicated when Port Address Translation (PAT) is used instead of NAT. PAT relies on the use of port numbers. Remember, these are TCP and UDP port numbers, stored within the TCP or UDP payload portion of the packets. If the payload is encrypted, the port numbers are encrypted too, and the PAT devices are unable to process the packets because they cannot access the port number. Even if the payload is not encrypted, the structure of IPSec packets is different from that of non-IPSec packets, and many PAT devices can't correctly parse the IPSec packet structure.
To resolve conflicts between IPSec and address translation, some IPSec implementations now support a feature called NAT Traversal (NAT-T). If both endpoints state during the IKE negotiation that they support NAT-T, they next check to see if a NAT or PAT device between them is altering either of their IP addresses or source ports. If address translation is in use, the endpoints move their IKE negotiations from UDP port 500 to 4500 and wrap all their ESP packets within UDP packets. Known as UDP encapsulation, this separates the IPSec information from the new outer UDP header, which is subsequently manipulated by the NAT or PAT device without any impact to the IPSec headers. Unfortunately, because standards for NAT-T are still not finalized, there may be interoperability problems between different types of endpoints.
VPN passthrough refers to the concept of successfully allowing VPN traffic to go through a perimeter defense device, even when that device performs NAT, PAT, or other actions that could adversely affect VPN traffic. VPN passthrough is often achieved by allowing VPN traffic to bypass functions such as NAT.
Whether you are implementing IPSec services on a VPN concentrator, a firewall, or a router, you have several options on where to place the IPSec services. This is discussed in detail in Chapter 12. Likely places for standalone VPN servers are on your perimeter, such as your DMZ or a screened subnet, and in parallel with your Internet firewall. Your design decision also depends on which IPSec architecture you want to implement.
As we discussed earlier in this chapter, three types of IPSec architectures exist. Each is appropriate to meet certain types of needs.