IPSec Overview

Cisco products support several tunneling protocols for VPNs, such as PPTP, L2TP, and IPSec. PPTP and L2TP are both popular tunneling protocols, but they require additional overhead and do not offer the same security enhancements that IPSec provides. For this very reason, Cisco (as well as this book) de-emphasizes PPTP and L2TP tunneling protocols. Thus, the following sections examine the different aspects of IPSec protocols and how they can be utilized to provide a secure pathway between pairs of gateways and other gateways or hosts.

What Is IPSec?

It is a common misconception that IPSec is a secure protocol unto itself. In all actuality, it is a framework of protocols that operate at the Network Layer (Layer 3) of the OSI model. IPSec is used to offer security services for IP traffic and establish tunnels between VPN gateways and hosts. Specifically, the security services that IPSec can provide are confidentiality, data integrity, authentication, and anti-replay protection.

The 642-511 exam expects you to understand the services that IPSec can provide and the aspects of each of those services.

IPSec Data Confidentiality

One of the most prominent features of IPSec, data confidentiality, is synonymous with IPSec's capability to encrypt data before traversing the shared network. If you think back to the briefcase example, the contents of the briefcase are scrupulously guarded from any form of prying eyes. With IPSec encryption, you can make it extremely difficult for eavesdroppers to interpret any intercepted data communications.

Encryption is achieved by employing several different algorithms (a mathematical function) which, in turn, are composed of different lengths (strengths) of keys. Clear text data is input into the algorithm and combined with the key to produce an encrypted text called cipher text. Unless you have the appropriate decrypting key, you cannot effortlessly compromise the contents of the data.

Encryption Keys

Encryption keys are strings of digits of varying lengths that are used by the encryption algorithms to produce cipher text. The longer the length of the keys, the more difficult the encryption is to decipher; however, longer keys require extra processing power and time. Common key lengths are 56 bits, 128 bits, and 160 bits, although key lengths have been known to be up to 2048 bits and beyond.

Encryption keys can be one of two types:

• Symmetric keys Symmetric keys are characterized by the use of the same key on both sides of the tunnel to encrypt and decrypt the data. They are kept private because these keys should never be transmitted across a public infrastructure or they might be intercepted. In the briefcase example, this would be the same key that locks and unlocks the handcuffs.

• Asymmetric keys Asymmetric keys utilize a public/private key pair. What one key encrypts the other key can decrypt. Data is encrypted with the public key (which can be publicly known) and can only be decrypted by its corresponding private key (which is secret and not shared).

Figure 2.2 depicts the differences between the two types of encryption keys. Notice in the top illustration that the symmetric keys are identical to encrypt and decrypt the data. The bottom diagram represents asymmetric encryption, which utilizes two separate keys a public key for encryption and the private key for decryption.

Figure 2.2. Symmetric and asymmetric encryption.

Because they require little processing power and are relatively fast, symmetric keys are typically utilized in encrypting and decrypting bulk data. Because symmetric keys need to be known by both parties for encryption and decryption, there must be a method of exchanging keys before the actual encryption takes place. Symmetric keys can be exchanged either manually or dynamically. Manual key exchange entails administrators exchanging the keys, usually by some out-of-band means such as via a secure phone call or mail carrier. Because this can be a daunting task, symmetric keys can also be exchanged dynamically by a method known as the Diffie-Hellman exchange.

It is not necessary to understand the complex mathematical calculations for the Diffie-Hellman exchange. It is more pertinent to understand the utility of Diffie-Hellman and which groups Cisco supports.

To dynamically obtain symmetric keys for bulk encryption in a secure manner, the sender has to encrypt keying materials to produce the symmetric keys with yet another encryption key called the secret key. This secret key is a mathematical derivative of the peer's declared public key and the local private key. Figure 2.3 illustrates this concept by displaying two IPSec peers that are initiating an IPSec tunnel. Both sides exchange their public keys which are mathematically combined with their local private keys. The output of this calculation is an identical secret key which can be used to secure subsequent messages, including device-level authentication and the exchange of keying material to generate symmetric keys for bulk encryption. The latter is depicted in Figure 2.3 as a locked safe which is secured and opened with the matching secret key.

Figure 2.3. Diffie-Hellman key exchange.

Diffie-Hellman is the key exchange method over insecure channels in which both sides of the IPSec session formulate the matching secret key. In addition, this shared key is never sent across the tunnel, thus forcing any eavesdropper to require each individual integer used in the mathematical calculation of that key to compromise the transfer. However, it is possible for attackers to launch a man-in-the-middle attack if they are capable of intercepting both sides' public keys during the initial exchange.

Cisco supports four groups of Diffie-Hellman exchange: Diffie-Hellman Group 1 (768 bit), Diffie-Hellman Group 2 (1024 bit), and Diffie-Hellman Group 7 (variable bit), which uses elliptical curve cryptography (ECC) for small processor devices that use Certicom movian wireless clients (described in the following section). Later software releases for the VPN 3000 Concentrator also support 1536-bit Diffie-Hellman group 5.

Encryption Algorithms

Numerous encryption algorithms are used today for encryption. For the purposes of this book, we will focus on the four that are employed by Cisco: DES, 3DES, RSA, and AES encryption algorithms.

Data encryption standard (DES) is a long-standing encryption algorithm that has been around since the late 1970s. DES comprises a 56-bit key which is applied to 64-bit blocks of data. DES was once considered a strong encryption algorithm; however, super-computers can break the DES encryption in a relatively short time. In response to this cryptographic possibility, 3DES was created to offer roughly 2⁵⁶ times more strength than encryption over DES. It does this by encrypting, decrypting, and then encrypting once again with three independent 56-bit keys (aggregate of 168 bits). Both DES and 3DES are symmetric algorithms, which means that they use the same symmetric key to encrypt and decrypt data.

The U.S. National Institute of Standards and Technology (NIST) proposed a challenge to form an algorithm standard that could replace DES and perhaps 3DES. The winner of this contest was the Rijndael algorithm, which the NIST implemented as the Advanced Encryption Standard (AES) algorithm in 2000. Similar to DES and 3DES, AES is a symmetric algorithm that uses matching keys to encrypt and decrypt data. AES, however, supports variable-length keys of 128, 192, and 256 bits to variable-length blocks of data, making it versatile for different IPSec implementations.

RSA is an asymmetric encryption algorithm that was named after its creators Ron Rivest, Adi Shamir, and Leonard Adleman. RSA's key length varies in size depending on the level of encryption security (and processing) you want to use. Data encryption is performed by combining clear-text data with the receiver's public key. With this method, only the receiver can decrypt the data with its own private key. Although RSA can be used for encryption and decryption of data, RSA is quite often used for creating digital signatures (discussed in Chapter 5).

Another noteworthy encryption algorithm is Elliptical Curve Cryptography (ECC). This asymmetric algorithm was created by Certicom and is used quite often in devices with low processing power. ECC is capable of speeding up asymmetric operations because of its remarkable mathematical algorithm, which makes it ideal for wireless clients such as the Certicom movian wireless client.

IPSec Data Integrity

IPSec data integrity is maintained by a hash algorithm to ensure that the data was not manipulated in transit by a man-in-the-middle attack. These algorithms take the data and combine it with a shared secret key to calculate a smaller fixed-length representation of the actual message (called a message digest). This whole hashing mechanism is known as a hashed message authentication code (HMAC). If any of the data payload has been changed in transit, it is reflected when the receiving party performs the same algorithm and the results do not match. Cisco supports a keyed HMAC variant of the two hashing algorithms Message Digest 5 (MD5) and Secure Hash Algorithm-1 (SHA-1). MD5 utilizes a 128-bit key in the hashing algorithm and produces a 128-bit message digest, whereas SHA-1 offers more cryptographic strength with a 160-bit secret key, which produces a 160-bit message digest.

IPSec Authentication

When the recipient of the briefcase receives the package, how can he be sure that it actually came from the true original sender? Perhaps if the sender signed his name to the contents of the briefcase or used a secret handshake of some sort, the recipient could validate the sender. Similarly, when establishing a VPN tunnel (before the tunnel is considered secure), you have to ensure that the device at the far end of the tunnel is truly the one which you mean to connect. This machine-level authentication is necessary because Diffie-Hellman exchanges are susceptible to man-in-the-middle attacks. The three methods that Cisco accepts to authenticate peers are preshared keys, RSA signatures, and RSA-encrypted nonces.

Preshared keys are characterized by having identical keys at each end of the tunnel. This key has to be manually configured in both devices that terminate the tunnel. The local device performs a keyed hash of data with the preshared key and sends the resulting message digest along with the data to the remote peer. When the remote peer uses the same preshared key to perform the hash and the message digest values match, the local device is authenticated. The reverse must also transpire before both sides can truly be considered authenticated. Although they are relatively easy to configure, preshared keys do not scale well in large VPN environments, because you would have to maintain a preshared key for each VPN peer with which you have to communicate. If the preshared key is somehow compromised, the reconfiguration involved could be significant.

Preshared keys fall into three different types: unique, group, and wildcard. Unique preshared keys utilize IP addresses, in addition to the key, for authentication. This is not a suitable design for remote access in which remote-end dynamic IP addresses constantly change. In such instances, group preshared keys would be appropriate because they tie a group name as opposed to an IP address for authentication. The last preshared key type, called wildcard preshared keys, does not use IP addresses or groups for authentication data. This is the least secure of the preshared keys because an authenticating device does not require any other proof aside from the preshared key.

RSA Digital Signatures utilize the RSA asymmetric algorithm discussed earlier. In contrast to normal encryption, digital signatures are created by digitally signing a set of data with the sender's private key. A hash of identity information is digitally signed with the sender's private key and is sent to the peer device, along with a digital certificate containing the local device's public key. The local device is authenticated when the remote uses the public key in the digital certificate and validates the hash output with its own hash.

Cisco also supports another digital signature algorithm that is not as common as RSA. This algorithm, known as digital signature algorithm (DSA), is used only for digital certificate signing (as opposed to encryption). DSA is less susceptible to attacks than RSA and is typically used by the U.S. government.

A final authentication method that is unique to Cisco is to use RSA encrypted nonces. Similar to RSA digital signatures, both sides of the VPN tunnel contain a public and private key; however, the public key is exchanged only via a secure out-of-band channel. After this is complete, both sides of the tunnel generate a pseudorandom number (nonce), which is encrypted with the peer's public key. Both sides exchange this information and decrypt it with their private key. Finally, both sides hash that output and send it to their peer, to be validated by a locally performed hash of their own nonces.

IPSec Anti-Replay Protection

IPSec utilizes sequence numbers to ensure that there are no instances of packet duplication. If the packet sequence number is the same or less than the previous packet's, the packet is discarded and possibly logged.