IP Security Overview

IPSec defines a network security architecture that allows secure network transmissions for an enterprise while introducing a minimum of overhead. IPSec allows you to secure packets at the network layer. By performing services at the network layer, IPSec secures information in a manner that is transparent to users and to the protocols that lie above the transport layer. IPSec provides layer 3 protection.

The IPSec architecture exercises an end-to-end security model. Only the endpoints of a communication need to be IPSec-aware. Computers and devices that serve as intermediaries of message transfer do not need to be IPSec-enabled. This allows a network administrator to implement IPSec for end-to-end security over diverse network infrastructures, including the Internet. Network connectivity devices such as bridges, switches, and routers can be oblivious to IPSec without compromising its efficacy. This end-to-end capability can be extended to various communication scenarios, including:

• Client-to-client

• Gateway-to-gateway

When IPSec is used to protect communications between two clients—for example, on the same LAN—the machines can utilize IPSec in transport mode. In transport mode, both clients must use TCP/IP as their network protocol. In this example, the endpoints of the secure communication are the source machine and the destination host.

In contrast, with a gateway-to-gateway solution, information traversing a transit network (such as the Internet), is protected by IPSec. Packets are protected as they leave the exit gateway and then decrypted or authenticated at the destination network's gateway. In this scenario, the host and destination computers do not have to employ IPSec and can use any LAN protocol supported by IPSec (such as Internetwork Packet Exchange/Sequenced Packet Exchange [IPX/SPX], AppleTalk, NetBEUI, or TCP/IP).

When gateways represent the endpoints of secure communication, IPSec works in tunnel mode. In tunnel mode, a tunnel is created between the gateways, and client-to-client communications are encapsulated in the tunnel protocol headers. You can create tunnels using IPSec as the tunneling protocol, or you can combine IPSec with Layer 2 Tunneling Protocol (L2TP). In the latter case, L2TP rather than IPSec creates the tunnel, and IPSec provides the encryption.

Exam Warning 

You must know the differences in the transport mode and tunnel mode implementations of IPSec.

Overview of IPSec Cryptographic Services

IPSec ensures security of communications by employing a variety of cryptographic techniques. Cryptography encompasses encryption, which is the making and deciphering of hidden or scrambled messages in such a manner that if the message or communication is intercepted, the thief cannot easily ascertain the contents of the message.

A good security system has several component features. The IPSec security architecture is designed to provide the following:

• Message integrity

• Message authentication

• Confidentiality

Exam Warning 

Ensure that you are familiar not only with the benefits that IPSec provides, but also with how it works to provide them for you.

Message Integrity

The term integrity refers to the assurance that the message received is identical to the message that was sent. Integrity is violated if a communication is somehow altered between the sending and receiving computers. Message integrity can be assured via the creation of digital signatures. A digital signature is like an electronic fingerprint. This fingerprint is a representation of a document's content. If an intruder were to capture the message in transit and change its contents, they would leave on the message a fingerprint that is different from the original fingerprint. The destination machine would detect that other hands had touched the document and therefore would consider the document's content invalid. Hash functions can be used to create the original fingerprint.

Hashing Messages

A message can be hashed by running it through a hashing algorithm. A key (a variable called a hash value) is used together with the hashing algorithm to create a hash so that only computers that know the key can create the same hash output of a message. The hashed output is always the same length; the algorithm creates fixed-length outputs from variable-length messages. This hashed output is often referred to as a message digest or a hash signature. You cannot reverse-engineer a digest to get the original message. Each packet must have a different hashed result.

For example, if I send you a message that states, "Cash the check," I will hash the message using a secret key that only you and I know about. After sending "Cash the check" through the hash algorithm using the secret key, we get a message digest of 12345.

Now I will send you the message, together with the message digest. In order to make sure that the original message was "Cash the check," you will send the contents of the message through the same hash algorithm and check the result. If you get 12345, it matches the digest sent to you. You know that indeed "Cash the check" was the original content of the message.

If a man in the middle had intercepted the message, they might have changed the content of the message to state "Don't cash the check." When you received the message, it would read "Don't cash the check." You would then run "Don't cash the check" though the hash algorithm, and the result would be 12389. This result does not match the message digest included with the message. Therefore, you know that the integrity of the message has been violated and should not be considered valid.

These message digests are also known as hash message authentication codes (HMACs). To derive an HMAC, Microsoft's implementation of IPSec uses one of two algorithms:

• Message Digest 5 (MD5)  This algorithm was developed by Ron Rivest of MIT and is defined in RFC 1321. MD5 processes each message in blocks of 512 bits. The message digest ends up being 128 bits.

• Secure Hash Algorithm (SHA1)  This algorithm also processes messages in blocks of 512 bits. However, the resulting message digest is 160 bits long. This confers a greater degree of security but is a bit more processor intensive and is therefore slower than MD5.

A shared secret key is required to make the hash method work. To ensure the validity of the secret key, you must utilize other technologies, such as a public key infrastructure, which can also take advantage of asymmetric key exchange to provide a higher degree of assurance.

Message Authentication

Authentication is concerned with establishing the identity of a sender or recipient. Integrity concerns itself with making sure that the message has not changed during transit. Authentication focuses on confirming the identities of the conversation participants. It would be of little value to receive a message of uncompromised integrity from an imposter.

IPSec uses three methods to carry out message authentication:

• Preshared key authentication

• Kerberos authentication

• Public key certificate-based digital signatures

Preshared Key Authentication

Preshared key authentication schemes depend on both members of the communication having preselected a secret key that will be used to identify them to each other. Data leaving the sending computer is encrypted with this agreed-upon key and decrypted on the other end using the same key.

Both members of the communication assume that if the other side has access to this preselected key, both members are who they claim they are. This type of authentication is accomplished in the following way:

1. The sending computer can hash a piece of data (called a challenge) using the shared key and forward this hashed data to the destination computer.

2. The destination computer receives the challenge and performs a hash using the same secret key. It then sends this hashed data back to the first sending computer.

3. If the hashed results are identical, the computers share the same secret key and are thus authenticated.

Even though preshared keys are effective in authenticating that each member has access to the same shared secret, this solution is not easily scalable. The shared secret must be manually keyed into the IPSec policy, which is not an issue if the same policy applies to the entire domain tree, but can become cumbersome when subdomains, organizational units, and individual machines require varying IPSec policies.

Kerberos Authentication

The Kerberos authentication method is also based on the shared secret principle. In this case, the shared secret is a hash of the user's password. For more information about the Kerberos authentication protocol, see the Chapter 8.

Public Key Certificate-Based Digital Signatures

As mentioned earlier in this chapter, a message digest is a hash of a message's contents. The combination of a key and a hash algorithm is used to create a message digest. A digital signature is an encrypted message digest. A message is authenticated when the digest is first decrypted, and then the decrypted hash must match the hash derived at the destination host.

The sending computer uses its private key to complete this process. Public key-based authentication is based on the principle that each computer has a public and private key pair created for it in advance. The public key is freely available to anyone who wants it; the private key is available only to the computer that owns it. For a public key infrastructure to work, the private key must be kept private. If a private key is compromised, all messages from that computer should be considered suspect and possibly originating from an imposter. A viable public key infrastructure includes these elements:

• Private keys that are kept absolutely secret

• Public keys that are freely available to anyone

• A trusted third party to confirm the authenticity of the public key (to ensure that the public key does indeed belong to the party who claims it)

The trusted third party is known as a certification authority (CA). The CA is required to digitally sign each party's public key to prevent attackers from providing a public key that they claim is that of another person but is in fact not the public key of the person they are impersonating.

A CA will digitally sign each user's public key. In this way, if someone sends you their public key, you can be sure that it is theirs, because a trusted third party has confirmed their identity and signed their public key. Here are two scenarios that illustrate the need for digital certificates and digital signatures: In the first scenario, say your boss wants to authenticate your identity using your public key. One way she can do this is by sending you a challenge message, which you encrypt with your private key. You then send it back to her after you have encrypted it. She can then use your public key to decrypt the message. If the message that she decrypts is the same as the message that she sent you, she can confirm that indeed it was you with whom she was communicating.

The problem is that she received your public key from you, yourself. How does she know that you, and not someone impersonating you, sent her your public key? You can solve this problem by having a mutually trusted third party digitally sign your public key. Both you and your boss trust that the third party has verified the identity of anyone for whom the third party signs its public key.

Notes from the Underground…
Nothing in Life Is Perfect

No security method is foolproof, and this includes the use of certification authorities to verify identity. In early 2001, someone tricked VeriSign into believing that they were a legitimate Microsoft employee and received two digital certificates from VeriSign that were to be used for code signing (see www.microsoft.com/technet/security/bulletin/ms01-017.asp). What is even more interesting (or scary depending on how you look at it) is that this security failure went unnoticed for a month before VeriSign caught onto the problem and revoked the digital certificates.

So, if one of the pioneers in the digital certificate game can be tricked by some slick social engineering into giving out two high-level digital certificates, what are the odds that the same thing might not happen to you on your network? No matter how secure you think you are or how many layers deep your defensive posture is, always remember that an attacker may be one step ahead of you. Eventually every system can and will be penetrated if someone tries hard enough and has the capabilities to pull off the attack. This is why defense in depth is such a critical concept. The more layers of defense you have in your network, the better your odds are of keeping an attacker out, or at worst, detecting and tracking an attacker should they penetrate your defenses. Never stop thinking you have done enough to protect your network because nothing in life is perfect.

In the second scenario, say you want to be sure that your boss is who she says she is. You do not have her public key at this point, so you ask her to send it to you. She sends you her signed certificate (the certificate is essentially her public key signed by the trusted third party). You already have the public key of the trusted third party. You use the third party's public key to verify the signature on the certificate. You know that this verified key is her public key, which she sent you. You can now send a challenge to confirm that you are indeed communicating with your boss and not an imposter. Additionally, you can check the most recently published CRL to ensure that the certificate being presented is still valid and has not been revoked.

Public key authentication is used by IPSec when non-Kerberos-enabled clients need to be authenticated and no preshared key has been established. Public key authentication must also be used when using L2TP tunneling and IPSec. Preshared keys can be used between a Windows 2000 RAS and a third-party firewall product when the two are acting as gateways for a corporate wide area network (WAN).

Exam 70-124: Objective 3.1.2: Confidentiality

Neither integrity nor authentication is concerned with protecting the privacy of information. Confidentiality is a matter of keeping private information private. To ensure confidentiality of IPSec communications, data must be encrypted using an encryption algorithm.

Data Encryption Standard

The encryption algorithm most commonly used with Microsoft's implementation of IPSec is the Data Encryption Standard (DES) algorithm. DES has long been the U.S. government standard for encryption. The DES algorithm is an example of a symmetric encryption algorithm. A symmetric encryption algorithm has each side of a communication employ the same secret key for encryption and decryption. This is in contrast to a public key infrastructure, in which two different keys are used. The public key approach is referred to as asymmetric encryption.

DES works on 64-bit blocks of data. The DES algorithm converts 64 input bits from the original data into 64 encrypted output bits. DES starts with 64-bit keys, but only 56 bits are actually used in the encryption process. The remaining 8 bits are used for parity.

A stronger version of DES is also available for use in Windows 2000/XP IPSec. This version is called 3DES, or Triple DES. Triple DES processes each block three times, which increases the degree of complexity over that of DES.

Note 

Although not used for IPSec at this time, the Advanced Encryption Standard (AES) is the new standard for encryption implemented by the U.S. government. The U.S. government encryption standard is known as the Federal Information Processing Standard (FIPS). AES uses the Rijndael symmetric encryption algorithm. Effective May 26, 2002, AES became an official government standard.

Cipher Block Chaining

Because the blocks of data are encrypted in 64-bit chunks, there must be a way to chain these blocks together. The chaining algorithm defines how the unencrypted text, the secret key, and the encrypted text (also known as ciphertext) will be combined to send to the destination host. These chaining algorithms also solve another problem. If the same data is sent twice, both blocks would look the same. This knowledge could be used by a cryptanalyst in trying to figure out the content of a message.

To prevent each block from looking the same, DES can be combined with cipher block chaining (CBC). This DES-CBC algorithm makes each ciphertext message appear different by using a different initialization vector (IV), which is a random block of encrypted data that begins each chain. In this fashion, each message's ciphertext can be made to appear different, even if the exact same message is sent a hundred times.

Exam Warning 

Be sure that you are familiar with the various methods utilized in IPSec for integrity, authentication, and confidentiality. Expect to be tested on them and their various combinations on this exam.

Exam 70-124: Objective 3.1.3: IPSec Security Services

IPSec engages the following two protocols to implement security on an IP network:

• Authentication header (AH)

• Encapsulating Security Payload (ESP)

Test Day Tip 

Ensure that you know and understand the differences between the AH and ESP protocols that make up IPSec. Be aware of how each is used to make secure communications possible in both transport and tunnel modes of operation.

The AH

The AH ensures data integrity and authentication, and can be used to prevent replay attacks. The AH does not encrypt data and therefore provides no confidentiality. When the AH protocol is applied in transport mode, the AH is inserted between the original IP header and the TCP header, as shown in Figure 6.1. The entire datagram is authenticated using AH.

Figure 6.1: The Datagram after Applying the Authentication Header in Transport Mode

ESP

The ESP protocol can provide authentication, integrity, and confidentiality to an IP datagram. Authentication services are available with ESP, but the original IP header prior to application of the ESP header is not authenticated. The ESP header, in transport mode, is placed between the original header and the TCP header, as shown in Figure 6.2. Only the TCP header, data, and ESP trailer are encrypted. If authentication of the original IP header is required, AH and ESP may be combined and used together.

Figure 6.2: The Datagram after Applying the Encapsulating Security Payload Header in Transport Mode

Figures 6.1 and 6.2 demonstrate packet configurations when AH or ESP is used in transport mode. Transport mode is used when point-to-point communications are taking place between source and destination computers. AH and ESP can be applied at a gateway machine connecting the LAN to a remote network. In this case, tunnel mode is used.

In tunnel mode, an additional IP header is added that denotes the destination tunnel endpoint. This tunnel header encapsulates the original IP header, which contains the IP address of the destination computer. Figure 6.3 shows a packet constructed for tunnel mode.

Figure 6.3: A Datagram with ESP Header in Tunnel Mode

Security Associations and IPSec Key Management Procedures

Two concepts you need to understand when looking at how IPSec works are security associations and key management. In the following sections, we look at both of these concepts.

Security Associations

When two computers establish a connection using IPSec, they must come to an agreement regarding which algorithms and protocols they will use. A single security association (SA) is established for each link a computer maintains with another computer via IPSec. A security association consists of several parts: a destination address, a security protocol, and a security parameters index (SPI), which is a unique identifier. If a file server has several simultaneous sessions with multiple clients, a number of SAs will be defined, one for each connection via IPSec.

Each security association has associated with it these parameters:

• Encryption algorithm (DES or 3DES)

• Session key (via Internet Key Exchange [IKE])

• Authentication algorithm (SHA1 or MD5)

A security parameters index (SPI) tracks each SA. The SPI is a value that uniquely identifies each SA as separate and distinct from any other IPSec connections current on a particular machine. The index itself is derived from the destination host's IP address and a randomly assigned number. When a computer communicates with another computer via IPSec, it checks its database for an applicable SA. It then applies the appropriate algorithms, protocols, and keys and inserts the SPI into the IPSec header.

An SA is established for outgoing and incoming messages, which means at least two security associations are necessary for each IPSec connection. In addition, a single SA can be applied to either AH or ESP, but not both. If both are used, two more security associations are created—one SA for inbound and one SA for outbound communications.

IPSec Key Management

Keys must be exchanged between computers in order to ensure authenticity, integrity, and confidentiality. Key management defines the following:

• Procedure used to determine how keys are formed

• Strength of keys

• How often keys are changed

• When keys expire

The establishment of a shared secret key is critical to secure communications. A shared secret can be manually established using the prearranged key method, but this technique does not scale well due to its inherent lack of flexibility.

Automated key management is the preferred method of key exchange. Automated key management uses a combination of the Internet Security Association Key Management Protocol and the Oakley Protocol (ISAKMP/Oakley Protocol). This combination of protocols is often referred to collectively as the IKE. The IKE is responsible for exchanging key material (groups of numbers that will form the basis of new keys), session keys, SA negotiations, and authentication of peers participating in an IPSec interaction.

IKE takes place across two phases: Phase 1, in which the two computers agree on mechanisms to establish a secure, authenticated channel; and Phase 2, in which SAs are negotiated for security protocols, using AH, ESP, or both.

The first phase establishes what is called the ISAKMP security association (ISAKMP SA), and the second phase establishes the IPSec SA.

Phase 1: Establishing the ISAKMP SA

The following processes take place during the ISAKMP SA phase:

1. The computers establish a common encryption algorithm, either DES or 3DES.

2. A common hash algorithm, either MD5 or SHA1, is agreed upon.

3. An authentication method is established. Depending on policy, this method can be Kerberos, public key encryption, or a prearranged shared secret.

4. A Diffie-Hellman group is agreed upon in order to allow the Oakley Protocol to manage the key exchange process. Diffie-Hellman provides a mechanism for two parties to agree on a shared master key, which is used immediately or can provide keying material for subsequent session key generation. Oakley Protocol determines key refresh and regeneration parameters.

Phase 2: Establishing the IPSec SA

After a secure channel is established by the creation of the ISAKMP SA, the IPSec SAs are established. The process is similar, except that a separate IPSec SA is created for each protocol (AH or ESP) and for each direction of transmission (inbound and outbound). Each IPSec SA must establish its own encryption algorithm, hash algorithm, and authentication method.

One important difference is that each IPSec SA uses a different shared key than that negotiated during the ISAKMP SA. Depending on how the policy is configured, the IPSec SA repeats the Diffie-Hellman exchange or reuses key material derived from the original ISAKMP SA. All data transferred between the two computers takes place in the context of the IPSec SA.

Exam Warning 

You should have a solid understanding of how the ISAKMP/Oakley Protocol SA process is conducted and carried out. Make sure you understand what is accomplished during each phase of the SA process.