New IPsec Troubleshooting Features

There are two new IPsec troubleshooting features you can use in the IOS:

• IPsec VPN Monitoring: IOS 12.3(4)T
• Invalid Security Parameter Index Recovery: IOS 12.3(2)T

The following three sections will discuss both of these features.

IPsec VPN Monitoring Feature

IPsec VPN monitoring is a feature new in IOS 12.3(4)T. This feature allows you to monitor VPN sessions to provide for enhanced troubleshooting. These enhancements include:

• Adding a description to IKE peers so that it becomes easier to identify the peer other than using their IP address or FQDN.
• Clearing a crypto session: before IOS 12.3(4)T, you had to clear both the Phase 1 and 2 connections to a peer individually to tear down the crypto session; in IOS 12.3(4)T, you can tear down both sets of connections with a single command.

The following two sections will discuss these enhancements.

Configuring IKE Peer Descriptions

To configure IKE peer descriptions, use the following configuration:

Router(config)# crypto isakmp peer {address peer_IP_address |
 hostname peer_hostname}
Router(config-isakmp-peer)# description description

You first must specify the identity of the peer (based on the configuration of the crypto isakmp identity command), which takes you into a subcommand mode. The description command allows you to assign a 80-character description, including spaces, for the remote peer. This description will then appear in the output of various show commands.

Note

If multiple remote peers sit behind the same PAT device, you cannot use address as an identity type for a description, since they'll all have the same IP address.

 

Seeing Peer Descriptions in show Commands

There are two show commands that take advantage of the use of descriptions:

• show crypto isakmp peer [IP_address_of_peer] Briefly displays the IPsec peer connections and descriptions. Example 19-17 illustrates the use of this command, where the local peer (192.1.1.40) is connected to the remote peer 192.1.1.42. A description of "Connection to SiteA" was assigned to this peer.

  Example 19-17. Using the show crypto isakmp peer Command

  RTRA# show crypto isakmp peer
  Peer: 192.1.1.42 Port: 500 Local: 192.1.1.40
   Description: Connection to SiteA
   Phase1 id: 192.1.1.42
  

• show crypto session[local local_IP_address] [remote remote_IP_address] [detail] Displays status information for active crypto map sessions. Example 19-18 illustrates the use of this command without the detail parameter and 19-19 with it. In Example 19-18, you can see the peer the router is connected to (192.1.1.42) and that the management and two data connections were built. Table 19-1 explains the statuses that can appear in the output of this command. If no flow exists, this could be because a dynamic crypto map is being used. Example 19-19, with the detail parameter, shows more information, including the description and the number of packets encrypted and decrypted.

Example 19-18. Using the show crypto session Command

RTRA# show crypto session
Crypto session current status
Interface: Ethernet0/0
Session status: UP-ACTIVE
Peer: 192.1.1.42 port 500
 IKE SA: local 192.1.1.40/500 remote 192.1.1.42/500 Active
 IPSEC FLOW: permit ip 192.168.2.0/255.255.255.0
 192.168.3.0/255.255.255.0
 Active SAs: 2, origin: crypto map

Table 19-1. Status of VPN Sessions

IKE SA	IPsec SAs	VPN Tunnel Status
Exists, active	Exists (flow exists)	UP-ACTIVE
Exists, active	None (flow exists)	UP-IDLE
Exists, active	None (no flow exists)	UP-IDLE
Exists, inactive	Exists (flow exists)	UP-NO-IKE
Exists, inactive	None (flow exists)	DOWN-NEGOTIATING
Exists, inactive	None (no flow)	DOWN-NEGOTIATING
None	Exists (flow exists)	UP-NO-IKE
None	None (flow exists)	DOWN
None	None (no flow exists)	DOWN

 

Example 19-19. Using the show crypto session detail Command

RTRA# show crypto session detail
Crypto session current status
Code: C - IKE Configuration mode, D - Dead Peer Detection
K - Keepalives, N - NAT-traversal, X - IKE Extended Authentication
Interface: Ethernet0/0
Session status: UP-ACTIVE
Peer: 192.1.1.42 port 500 fvrf: (none) ivrf: (none)
 Desc: Connection to SiteA
 Phase1_id: 192.1.1.42
 IKE SA: local 192.1.1.40/500 remote 192.1.1.42/500 Active
 Capabilities:D connid:2 lifetime:23:48:48
 IPSEC FLOW: permit ip 192.168.2.0/255.255.255.0
 192.168.3.0/255.255.255.0
 Active SAs: 2, origin: crypto map
 Inbound: #pkts dec'ed 9 drop 0 life (KB/Sec) 4479991/2929
 Outbound: #pkts enc'ed 9 drop 0 life (KB/Sec) 4479991/2929

Tip

I prefer to use the show crypto isakmp peer command over the show crypto isakmp sa command because the former gives me a brief description of the connection. I also prefer to use the show crypto session command over the show crypto ipsec sa command because the former easily summarizes the important information in a short display. The latter display is too verbose for me for a quick determination of whether either the Phase 1 or 2 SAs have been established.

 

Clearing Crypto Sessions

Another new feature introduced with the IKE description feature is the ability to delete all IKE and IPsec SAs associated with all peers or a specific peer. Before this enhancement, you had to delete the management and data SAs individually. Use the following command to delete all SAs associated with a peer or peers:

Router# clear crypto session
 [local local_IP_address [port local-port]]
 [remote ip-address [port remote-port]]

When you execute this command, the session(s) torn down will have "DOWN-NEGOTIATING" as the status in the output of the show crypto session command, indicating that the SAs are either completely down or in the process of being brought back up.

Invalid Security Parameter Index Recovery Feature

The invalid security parameter index (SPI) recovery feature was introduced in IOS 12.3(2)T and integrated into 12.2(18)SXE. This feature allows the router to recover from an invalid security parameter index error (displayed as Invalid SPI in the output of the debug crypto ipsec command. With this feature, the IPsec peers can resynchronize their SA databases and successfully bring up the data connections. The following two sections will discuss more information about how an invalid SPI condition can occur and how to enable the feature.

Invalid SPI Condition and the Invalid SPI Recovery Feature

An invalid SPI condition can occur if one IPsec peer dies (is shut down, is rebooted, has its interface reset, loses its management connection to a peer, and so on) and has an existing IPsec session to a remote peer. The remote peer still might try to use the SA even though a new one is built (with a new SA). The old SPI value from the old SA is no longer in the local peer, and the peer will respond with an "INVALID SPI NOTIFY" message to the remote peer, indicating that the data SA that the remote peer is trying to use is no longer available. The local peer's default action is to continue dropping traffic from the invalid SA (commonly referred to as a "black hole").

Upon receiving the "INVALID SPI NOTIFY" message, the default behavior of the local peer is to drop traffic. With the recovery feature enabled on both routers, the remote router will understand that an abnormal condition occurred with the local peer and that the remote peer should delete the existing SAs and establish new ones.

Note

Dead Peer Detection (DPD) can also detect this problem; however, DPD might take longer to detect it, depending on the keepalive interval. The Invalid SPI Recovery feature is more similar to the Initial Contact supported by Easy VPN devices.

 

Invalid SPI Recovery Configuration

To enable the invalid SPI recovery feature, use the following command:

Router(config)# crypto isakmp invalid-spi-recovery

This should be configured on all IOS routers that have peer relationships. Once enabled, you can use the debug crypto ipsec and show crypto ipsec sa commands to verify that the feature is enabled. When an invalid SPI condition exists, you'll see a message similar to Example 19-20, where the destination and source addresses are replaced by the peer addresses. In this example, a data SA using ESP was determined to be invalid.

Example 19-20. Message from an Invalid SPI Condition

%CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has
 invalid spi for destaddr=x.x.x.x, prot=50,
 spi=0x1214F0D(18960141), srcaddr=y.y.y.y

Tip

To test the configuration of the invalid SPI recovery feature, from the local peer, bring up an IPsec session to a remote peer (if one doesn't exist). On the local peer, execute the debug crypto ipsec command. Clear the Phase 1 and 2 SAs on the remote peer. From the local peer, send traffic to the remote peer. A small number of packets might be dropped, but on the remote peer you should see the message in Example 19-20 and on the local peer you should see the old data SAs torn down from the debug output and new ones built.

Caution

A denial of service (DoS) attack can occur if the invalid SPI recovery is enabled. The feature has a built-in mechanism to reduce the likelihood of this occurring, but there is still a chance that a DoS can occur; therefore, Cisco has disabled this feature by default, requiring you to enable it manually.