Packet Authentication

Packet authentication is used for two purposes:

  • To provide data origin authentication
  • To detect packets that have been tampered with

The following sections will discuss how packet authentication works (implementation), examples of functions that are used with packet authentication, the uses of packet authentication, and issues with performing packet authentication.

Packet Authentication Implementation

Hashing functions are used to create a digital signature by taking a variable-length input, such as user data or a packet, along with a key and feeding it into a hashing function. The output is a fixed-length result. If the same input is fed into the hashing function, it will always result in the same output.

Hashing Message Authentication Codes (HMAC) functions are a subset of hashing functions. HMAC functions were developed specifically to deal with authentication issues with data and packets. HMACs use a shared secret symmetric key to create the fixed output, called a digital signature or fingerprint. Generic hashing functions have a drawback: If an eavesdropper can intercept the sent data, he can easily generate his own signature of the data and send this "doctored" information to you. HMAC overcomes this problem by using a shared secret key to create the digital signature; therefore, only the parties that know the key can create and verify the signature for sent data.


With HMAC functions, if the same data and same secret key are used to generate a signature, they will always generate the same signature; if you change the data or the key value, the resulting signature changes.

Two examples of HMAC functions are MD5 and SHA, which I'll discuss in the following two sub-sections. For now, know that the basic mechanics for HMAC functions are shown in Figure 2-2. Here is an explanation of the steps shown in Figure 2-2:


The source takes data that needs to be protected and a key, which is shared with the destination, and runs it through an HMAC function.


The output of the HMAC function is a digital signature or fingerprint. In IPsec VPNs, it is also called an Integrity Checksum Value (ICV).


The source then takes the data that was originally fed into the HMAC function and sends the data along with the digital signature, to the destination.


The destination will use the same process to verify the signature; it takes the data sent by the source, along with the same shared key, and inputs this into the same HMAC function, resulting in a second signature.


The destination then compares the source's sent signature to the just-computed signature. If they are the same, the destination recognizes that the only device that could have created the signature was a device with the same key. The HMAC function is a one-way process: it is impossible to reverse-engineer the process by taking the fingerprint and data and coming up with the symmetric key used to create the fingerprint. If the two fingerprints are different, the destination assumes that the data was tampered with (purposefully, like by an attacker, or by accident, like data corruption or address translation) and discards the data.

Figure 2-2. HMAC Signature Creation and Verification

Many VPN implementations use HMAC functions to verify that packet contents haven't been tampered with, such as IPsec's Authentication Header (AH) protocol and the Encapsulation Security Payload (ESP) protocol. The following two sections will discuss two of the most popular HMAC functions: MD5 and SHA.

MD5 HMAC Function

Message Digest 5 (MD5) was developed by Ronald Rivest in 1994. It is specified in IETF RFC 1321. MD5 creates a 128-bit signature (16 bytes in length). It is faster, but less secure, than SHA. MD5 is probably the most popular HMAC function used in the security market today. You can find it being used in PPP's Challenge Handshake Authentication Protocol (CHAP) and routing protocol authentication for routing protocols such as Border Gateway Protocol (BGP), Enhanced Interior Gateway Routing Protocol (EIGRP), Intermediate System-to-Intermediate System (IS-IS), Open Shortest Path First (OSPF), and Routing Information Protocol (RIP) Version 2.

SHA HMAC Function

The Secure Hashing Algorithm (SHA) was developed by NIST and specified in the Secure Hash Standard (SHS FIPS 180). A revision was made to the original HMAC function, called SHA-1, in 1994. It is detailed in the ANSI X9.30 standard. Using SHA-1 in VPNs such as IPsec is described in RFC 2404. SHA-1 creates a 160-bit signature (20 bytes in length). SHA-1 is slower than MD5, but more secure; because its signature is larger, it is stronger against brute force attacks to break the function and discover the shared secret key.


In early 2005, certain groups of people have proven that it is possible to create the same signature with different inputs than the original inputs in MD5, creating what is called a collision. This can create security issues because it is now possible, in MD5, to create false signatures on a very small scale. Because of this, most vendors support the use of SHA. However, there are reports that the current version of SHA also suffers from this problem. There has not been that much concern about this problem with VPN implementations, but there has with other applications. Because of this possible security issue, work is being developed on SHA-256 and SHA-512, which have much longer signature lengths (256-bit and 512-bit) than MD5 and SHA, making it extremely unlikely, with current computing technology, that collisions can be created purposefully for signatures.


Packet Authentication Uses

As a destination device, how can you verify that packets sent to you are coming from a trusted source which can be verified, versus packets being sent to you from an attacker? HMAC functions can be used for this verification process. We can use packet authentication, through HMAC functions such as MD5 and SHA, to verify that a packet is sent by a trusted source and that it wasn't tampered with while in transit.

As a simple example of using these functions for the verification process, I'll use authentication with a routing protocol. One concern for a router administrator is using a dynamic routing protocol and accepting routing updates from a valid source. You wouldn't want an attacker to corrupt your routing tables by sending your routers false routing information.

With HMAC functions, you can provide this authentication and verification function. With a routing protocol, a router can take routing information that you want to send to a neighboring router, along with an HMAC symmetric key, and run this through an HMAC function, producing a packet signature. The original routing information and packet signature are sent to the neighboring router. The neighboring router then takes the sent routing information and the shared HMAC key and runs them through the same HMAC function. If the computed packet signature matches what the source sent, the neighboring router recognizes that because the same key was used to create and verify the signature, the routing update must be from a trusted source. HMAC functions can provide the same advantage for VPN implementations and for other applications like routing protocol authentication.

Routing Protocol Authentication

In the early 1990s, I once worked on a project to implement and test Internet services that would make documents available to the public for one of the main departments of the state government of Florida. This was before web browsers existed and the applications available to me were FTP and Gopher.

While waiting for a permanent T3 connection from the local carrier (which took almost four months), the networking department of Florida State University (FSU) was kind enough to allow the department to connect to FSU's network for Internet access via a temporary fractional T1 connection. I set up my UNIX server for this access, connected it to FSU's network at about 4 pm, and then went home for the evening. The UNIX server was dual-connected: one connection went to FSU's network and one went to the state government network.

Unfortunately, I had forgotten that RIPv1 was enabled by default on my UNIX server, and both the state government and FSU's network was using RIP. In this instance, the state government was passing my UNIX box a default route, which, in turn, I passed into FSU's network. Unfortunately, a large part of FSU's network, instead of using its default route to reach the Internet, used the state government's default route, breaking FSU's Internet connectivity.

When I arrived to work the next morning, I had a not-so-nice voice mail from the administrator of FSU's network concerning my "default" route. I was red-faced about the problem and immediately disabled routing on my UNIX server to solve the problem. As you can see, without any type of authentication, routing updates can be very easily falsified by rogue devices such as my UNIX server.

One question that I'm commonly asked about VPNs is "Why would you need to perform packet authentication if you're already using encryption?" For example, obviously the only way to decrypt data successfully is if the destination has the encryption key that was used for the initial encryption. Even though that is true, there are still two issues. First, after decrypting the information, you would need to verify that what was decrypted was actually the information sent and not spoofed traffic. Second, a hacker could take advantage of this process by spoofing packets to your device, causing it to waste CPU cycles to decrypt the traffic.

Hashing functions require very few CPU cycles to create and verify the digital signature (hashed output). So they are more efficient. Therefore the following steps typically occur in protecting information between a source and destination in a VPN implementation, as shown in Figure 2-3 and detailed in the following step sequence:


The source encrypts the data with the encryption key and an encryption algorithm.


The source takes the encrypted data and hash key and feeds them into an HMAC function, resulting in a packet signature.


The source sends the encrypted data and the packet signature to the destination.


The destination takes the received encrypted data, along with the same hash key, and runs them through the same HMAC function as the source.


The destination compares the computed packet signature and the received packet signature.


If the just-computed signature matches what the source device sent it, then no packet tampering has occurred, and therefore the destination decrypts the encrypted data using the encryption key; otherwise the destination assumes the packet has been tampered with and discards the data.

Figure 2-3. Protection Steps


Packet Authentication Issues

There are three main issues with the use of HMAC functions:

  • How the shared secret key actually gets shared between the two peers
  • The impact that address translation devices have on data signatures created by HMAC functions
  • How VPNs implement HMAC functions

The following three sections will discuss each of these issues in more depth.

Sharing the HMAC Secret Key

HMAC functions have more than one problem, including: both devices that share and protect data need the same key. Somehow the symmetric key must be shared for the HMAC function the two devices are using. Of course, if you send the shared key across a public network, the key is susceptible to an eavesdropping attack.

One option is to hard-code the key on both devices; however, this option does doesn't scale well in large networks where a security policy states that keys should be changed periodically. Another option is to use an already encrypted connection across which to share the key. This assumes, though, that an encrypted connection exists between the peers. The "Key Exchange" section later in this chapter will explore this issue in more depth.

Sending Data and HMAC Signatures Through Translation Devices

Another problem with HMAC functions and digital signatures is that your signature could be corrupted by an intermediate device as your data is being sent between two devices. For example, an address translation device performing network address translation (NAT) or port address translation (PAT) alters the headers of IP packets or TCP/UDP segments. If these fields were part of the data fed into the HMAC function, and these fields were changed by the address translation device, the original digital signature would be corrupted, and the destination device would consider that the packet had been tampered with. Also, an intermediate device might place or change quality of service (QoS) information in a packet header.

One solution is to not include all fields in a packet when computing the digital signature with the HMAC function; for example, include only the user data, or certain fields of packet or segment headers that you know won't be changed by an intermediate device. Basically, you would have to exclude all mutable fields, or fields that had the possibility of being altered. These fields include the following in an IP packet: IP address fields, the time-to-live (TTL) field, the type of service (TOS) field, the TCP or UDP port number fields, and possibly other fields.

Using HMAC Functions in VPN Implementations

Even though HMAC functions commonly are used for authentication and integrity checking functions, one issue might arise with certain VPN implementations. I'll use IPsec as an example and one of its security protocols: AH (discussed in more depth in Chapter 3, "IPsec"). AH supports both the MD5 and SHA-1 HMAC functions.

Assume that AH needs to protect 200 bytes of data with SHA-1. SHA-1 creates a 160-bit signature, which is 20 bytes. The addition of the signature adds 10 percent overhead to the data transfer, assuming that the average packet size sent was 200 bytes. In addition, AH can use either MD5 or SHA-1, making the AH packet length variable, because MD5 adds 16 bytes for the signature and AH 20 bytes. IPsec's dilemma was to set the signature field to a fixed 12 bytes, meaning that AH and ESP include only the first 12 bytes no matter whether MD5 or SHA-1 is used.

This is a problem because if you use only part of the hashed output of the signature, there is more of a chance an attacker might find data that produces the same partial hash signature. For example, with MD5's signature being 16 bytes in length, it would take 2128 different guesses at messages before finding one that produced the same signature (a collision). If you decided to only use the first 8 bytes of the MD5 signature to reduce the size for efficiency, it might take an attacker 264 guesses; or assuming that the attacker had to go through only half the number of possibilities, then he would have to go through only 263 guesses to find the data that produced the same signature. Because IPsec has AH and ESP using 12 bytes for the signature, it might require the attacker to go through 296 possibilities to find the message.

Also, because IPsec isn't using all the bits of the signature, there is a chance, albeit a small chance, that the data sent might get corrupted. In addition, when the destination received the sent data, the destination might be able to duplicate the same signature and assume the corrupted data was valid. However, given that 12 bytes produces a result of 296 possibilities, it would be extremely rare to run into this situationyou would actually have a better chance of winning the state lottery than finding a corrupted packet that the destination found valid through the process of a truncated signature of 12 bytesbut winning the lottery would definitely be a lot more fun!


Just because a VPN implementation says it supports a particular security function, for example, AH and ESP supporting MD5 and SHA or 3DES using three separate keys, that doesn't mean that it implements these features the way they were intended. In AH's and ESP's case, they truncate the length of MD5 and SHA-1's signatures to a fixed length of 12 bytes, reducing the level of security natively provided by MD5 and SHA-1.

Part I: VPNs

Overview of VPNs

VPN Technologies




Part II: Concentrators

Concentrator Product Information

Concentrator Remote Access Connections with IPsec

Concentrator Remote Access Connections with PPTP, L2TP, and WebVPN

Concentrator Site-to-Site Connections

Concentrator Management

Verifying and Troubleshooting Concentrator Connections

Part III: Clients

Cisco VPN Software Client

Windows Software Client

3002 Hardware Client

Part IV: IOS Routers

Router Product Information

Router ISAKMP/IKE Phase 1 Connectivity

Router Site-to-Site Connections

Router Remote Access Connections

Troubleshooting Router Connections

Part V: PIX Firewalls

PIX and ASA Product Information

PIX and ASA Site-to-Site Connections

PIX and ASA Remote Access Connections

Troubleshooting PIX and ASA Connections

Part VI: Case Study

Case Study


The Complete Cisco VPN Configuration Guide
The Complete Cisco VPN Configuration Guide
ISBN: 1587052040
EAN: 2147483647
Year: 2006
Pages: 178
Authors: Richard Deal © 2008-2020.
If you may any questions please contact us: