Troubleshooting IS-IS Routing Problems

The discussions in the previous sections of this chapter focus on tools and commands, ranging from simple to complex, needed for troubleshooting IS-IS problems. These sections provide a good foundation for discussing actual problems and troubleshooting methodology. Common IS-IS routing problems fall under the following two broad categories:

• Adjacency formation problems

• Route maintenance problems

During normal operation, IS-IS routers form and maintain adjacencies with each other by using hello packets. Routing information is then exchanged by flooding LSPs, which are stored in appropriate Link-State databases (Level 1 or Level 2). Sequence number packets (CSNPs and PSNPs) provide control for the flooding process and ensure database synchronization. All these processes need to occur successfully to ensure accurate dissemination of routing information in the IS-IS domain. Any failures result in inconsistencies that ultimately result in routing problems. The following sections identify problems and list the corresponding ways to isolate those problems.

IS-IS Adjacency Formation Problems

Adjacency formation problems are common IS-IS failures. They mainly occur as a result of router misconfiguration, hardware and software failures, interoperability problems between different IOS Software releases, and interoperability problems between routers from different vendors . Adjacency problems are easier to isolate than routing problems. The following list of adjacency problems are discussed in this section:

• Mismatched Level 1 and Level 2 interfaces

• Misconfigured NSAPs

• Duplicate system IDs

• Mismatched MTUs

• Misconfigured IP addresses and Subnets

Mismatched Level 1 and Level 2 Interfaces

The default mode of operation for Cisco routers running IS-IS is Level 1-2. In this mode, a router can form both Level 1 and Level 2 adjacencies with neighbors in the same area and form only Level 2 adjacencies with neighbors in different areas. Figure 10-4 diagrams such a configuration. In the output shown in Example 10-23, RT1 forms a Level 1-2 adjacency with RT5, which is in the same area (49.0001), but forms only a L2 adjacency with RT2, which is in area 49.0002. The default Level 1-2 mode can be modified for all interfaces on the router by using the router configuration-level command IS-type or for a specific interface with the interface-level configuration command isis circuit-type level [ level-1 level-2 ]. If RT1 is misconfigured as a Level 1 only on Serial0/0 using any of the previously mentioned commands, it loses the adjacency with RT2. Consequently, the domain will be partitioned and there will be no communication between area 49.0001 and area 49.0002. This behavior is confirmed in Example 10-27.

Figure 10-4. Test network for studying mismatched levels of routing between connected interfaces.

Example 10-27 show clns neighbors During Normal Operation

 RT1#  show clns neighbors  System Id   Interface  SNPA              State  Holdtime  Type Protocol RT5         Et0/0      00d0.58eb.ff01    Up     26        L1L2 IS-IS RT2         Se0/0      *HDLC*            Up     23        L2   IS-IS 

Example 10-28 Simulating Mismatched Levels of Routing on a Seriel Link (refer to Figure 10-4

 RT1#  conf t  Enter configuration commands, one per line.  End with CNTL/Z. RT1(config)#  router isis  RT1(config-router)#  is-type level-1  RT1(config-router)#  ^Z  RT1#  show clns neighbors  System Id      Interface   SNPA             State  Holdtime  Type Protocol RT5            Et0/0       00d0.58eb.ff01   Up      26        L1   IS-IS RT2            Se0/0       *HDLC*           Up      280       IS   ES-IS 

In example 10-28, the show clns neighbors output shows that RT1 dropped the IS-IS adjacency with RT2 and shows instead an ES-IS adjacency in place. This is because the ES-IS adjacency process runs independently of IS-IS, and it is sustained by ESHs rather than IIHs. Also because RT1 was made globally a Level 1 router with the IS-type command, it has formed only a Level 1 IS-IS adjacency with RT5.

Misconfigured NSAPs ( NETs )

Each IS-IS node must have at least one NSAP address (NET) to identify it as a network node. This NET consists of the area ID of the node, the System ID, and a 0-value NSEL. The system ID is required to be unique within the area; the NSEL value is fixed at 0x00. The area ID (also referred to as the area prefix ) must be the same for all nodes in the same area. For nodes with multiple NETs, the system ID must be the same in all of them, and at least one of the area prefixes must be shared with another node in the same area. The effect of misconfiguring a NET is illustrated in Figure 10-5. In this example, RTE, RTF, and RTG are meant to be together in area 49.0001. They are also meant to form both Level 1 and Level 2 adjacencies.

Figure 10-5. Test network for studying misconfigured NSAP problem.

The outputs of the show clns neighbors command from RTE, RTF, and RTG shown in Example 10-29 indicate RTE formed only Level 2 adjacencies with RTF and RTG. However, RTF and RTG formed Level 1-2 adjacencies with each other as expected. This creates a suspicion about the configuration and operation of RTE.

Example 10-29 Troubleshooting Misconfigured NSAPs

 RTE#  show clns neighbors  System Id   SNPA             Interface   State  Holdtime  Type Protocol RTF         0000.0c76.f098   Et0         Up     27        L2   IS-IS RTG         0000.0c76.f096   Et0         Up     26        L2   IS-IS RTF#  show clns neighbors  System Id   SNPA              Interface  State  Holdtime  Type Protocol RTB         *HDLC*            Se0        Up     27        L2   IS-IS RTE         0000.0c76.f1fa    Et0        Up     9         L2   IS-IS RTG         0000.0c76.f096    Et0        Up     28        L1L2 IS-IS RTG#  show clns neighbors  System Id    SNPA              Interface   State  Holdtime Type Protocol RTE          0000.0c76.f1fa    Et0         Up     18       L2   IS-IS RTF          0000.0c76.f098    Et0         Up     24       L1L2 IS-IS 

A glance at the MAC addresses under the SNPA column of Example 10-29 shows that RTE has a higher MAC address than RTF and RTG, so with each node retaining the default interface priority value, RTE should be both the Level 1 and Level 2 DIS. This is confirmed by the show clns interface output of RTE, shown in Example 10-30. The value of the circuit ID gives that clue. In Example 10-31, RTF points correctly to RTE as the Level 2 DIS but incorrectly to itself as the Level 1 DIS. This implies that there is a problem with Level 1 communication. Because this is a new setup and all the defaults have not been tampered with, the only factor that might dictate the type of adjacency formed is the area prefix. Further inspection of the NET configured on RTE shows that it was misconfigured with 49.0002.0000.0000.0002.00 rather than 47.0002.0000.0000.0002.00. This area ID mismatch resulted in RTF and RTG forming only Level 2 adjacencies with RTE.

Example 10-30 Determinig the DIS on the LAN (refer to Figure 10-5)

 RTE#  show clns interface e0  Ethernet0 is up, line protocol is up   Checksums enabled, MTU 1497, Encapsulation SAP   ERPDUs enabled, min. interval 10 msec.   RDPDUs enabled, min. interval 100 msec., Addr Mask enabled   Congestion Experienced bit set at 4 packets   CLNS fast switching enabled   CLNS SSE switching disabled   DEC compatibility mode OFF for this interface      Next ESH/ISH in 4 seconds   Routing Protocol: IS-IS     Circuit Type: level-1-2     Interface number 0x0, local circuit ID 0x1  Level-1 Metric: 10, Priority: 64, Circuit ID: RTE.0  1     Number of active level-1 adjacencies: 0  Level-2 Metric: 10, Priority: 64, Circuit ID: RTE.0  1     Number of active level-2 adjacencies: 2     Next IS-IS LAN Level-1 Hello in 1 seconds     Next IS-IS LAN Level-2 Hello in 1 seconds 

Example 10-31 Confiramation of DIS Conflict

 RTF#  show clns interface  Ethernet0 is up, line protocol is up Ethernet0 is up, line protocol is up   Checksums enabled, MTU 1497, Encapsulation SAP   ERPDUs enabled, min. interval 10 msec.   RDPDUs enabled, min. interval 100 msec., Addr Mask enabled   Congestion Experienced bit set at 4 packets   CLNS fast switching enabled   CLNS SSE switching disabled   DEC compatibility mode OFF for this interface      Next ESH/ISH in 4 seconds   Routing Protocol: IS-IS     Circuit Type: level-1-2     Interface number 0x0, local circuit ID 0x1  Level-1 Metric: 10, Priority: 64, Circuit ID: RTF.0  1     Number of active level-1 adjacencies: 1  Level-2 Metric: 10, Priority: 64, Circuit ID: RTE.0  1     Number of active level-2 adjacencies: 2 

Duplicate System IDs in an Area

All IS-IS nodes in an area must have the same area prefix and a unique system ID. If a node has multiple NETs configured, each address must retain the same system ID. This is a critical protocol requirement, especially because the system ID forms part of the LSPID, and uniqueness is required to identify the owners of LSPs flooded within the area. When different nodes in the area are erroneously configured with the same system ID, the problem is detected and each node logs appropriate error messages to that effect (see Example 10-32). If the nodes are directly connected, each router immediately detects the problem as it exchanges hellos to establish adjacency. Consequently, the adjacency fails. If they are not directly connected, however, they overwrite each other's LSP for some time until the IS-IS process determines, based on the frequency of occurrence, that the problem is because of duplicate system IDs and logs appropriate error messages. Example 10-33 shows the output of the debug isis adj-packet command for a duplicate system ID situation between directly connected neighbors.

Example 10-32 Logging of Duplicate System ID Errors

 RT1#  show logging  Mar 10 16:41:20: %CLNS-3-BADPACKET: ISIS: LAN L1 hello, Duplicate system ID det) Mar 10 16:42:22: %CLNS-3-BADPACKET: ISIS: LAN L1 hello, Duplicate system ID det) Mar 10 16:43:21: %CLNS-3-BADPACKET: ISIS: LAN L1 hello, Duplicate system ID det) 

Example 10-33 Debugging Duplicate System ID Scenarios

 RT1#  debug isis adj-packet  Mar 10 16:41:53: ISIS-Adj: Sending L1 IIH on Ethernet0/0, length 1497 Mar 10 16:41:55: ISIS-Adj: Rec L1 IIH from 00d0.58eb.ff01 (Ethernet0/0), cir ty7 Mar 10 16:41:55: ISIS-Adj: Duplicate system id Mar 10 16:41:56: ISIS-Adj: Sending L1 IIH on Ethernet0/0, length 1497 Mar 10 16:41:58: ISIS-Adj: Rec L1 IIH from 00d0.58eb.ff01 (Ethernet0/0), cir ty7 Mar 10 16:41:58: ISIS-Adj: Duplicate system id Mar 10 16:41:59: ISIS-Adj: Sending L1 IIH on Ethernet0/0, length 1497 Mar 10 16:42:00: ISIS-Adj: Rec L1 IIH from 00d0.58eb.ff01 (Ethernet0/0), cir ty7 Mar 10 16:42:00: ISIS-Adj: Duplicate system id 

Mismatched Interface MTUs

ISO 10589 mandates the padding of transmitted hello packets, which are used to establish and maintain adjacencies, to the maximum possible data size a router can receive and process on an interface. This provides a packet-size negotiation mechanism, which ensures adjacency forms only between systems that can receive and process the largest possible data size that the other can transmit.

Cisco IOS Software adheres to this specification by padding hellos to the full MTU size of the interface. Consequently, in the default mode of operation, IS-IS routers running Cisco IOS Software do not form adjacencies if their physical interface MTU values do not match. Verification of possible MTU mismatch should, therefore, be considered when troubleshooting adjacency problems. Examples 10-34 and 10-35, which are based on Figure 10-6, illustrate adjacency failure resulting from an MTU mismatch. Figure 10-6 shows two routers connected back-to-back over a serial link. As seen in the debug output in Examples 10-34 and 10-35, the adjacency of the serial link is dropped at approximately 20:44:16 when the MTU on RT2 is changed from 1500 to 2000. From then on, RT2 pads its hellos to 1999, which are ignored by RT1. After three hellos are ignored by RT1, the hello holdtime expires , the adjacency is dropped, and an adjacency change event is logged.

Figure 10-6. Investigating an MTU mismatch problem.

RT1 retains an ES-IS adjacency with RT2; however, RT2 receives and processes the smaller 1499-byte hellos from RT1 and puts the IS-IS adjacency in init state, hoping to complete the three-way handshake to establish full IS-IS adjacency. The IS-IS adjacency is never completed as long as the two RT2's MTU remain larger than that of RT1.

Example 10-34 Debugging an MTU Mismatch on RT1

 RT1#  debug isis adj-packet  IS-IS Adjacency related packets debugging is on Mar 10 20:43:56: ISIS-Adj: Sending serial IIH on Serial0/0, length 1499 Mar 10 20:43:59: ISIS-Adj: Rec serial IIH from *HDLC* (Serial0/0), cir type L1L2 Mar 10 20:43:59: ISIS-Adj: rcvd state UP, old state UP, new state UP Mar 10 20:43:59: ISIS-Adj: Action = ACCEPT Mar 10 20:44:05: ISIS-Adj: Sending serial IIH on Serial0/0, length 1499 Mar 10 20:44:13: ISIS-Adj: Sending serial IIH on Serial0/0, length 1499 Mar 10 20:44:22: ISIS-Adj: Sending serial IIH on Serial0/0, length 1499  Mar 10 20:44:29: %CLNS-5-ADJCHANGE: ISIS: Adjacency to RT2 (Serial0/0) Down, ho  d Mar 10 20:44:29: ISIS-Adj: L2 adj count 0 Mar 10 20:44:31: ISIS-Adj: Sending serial IIH on Serial0/0, length 1499 Mar 10 20:44:40: ISIS-Adj: Sending serial IIH on Serial0/0, length 1499 Mar 10 20:44:48: ISIS-Adj: Sending serial IIH on Serial0/0, length 1499 Mar 10 20:44:57: ISIS-Adj: Sending serial IIH on Serial0/0, length 1499 Mar 10 20:45:07: ISIS-Adj: Sending serial IIH on Serial0/0, length 1499 RT1#  show clns neighbors  System Id    Interface   SNPA            State  Holdtime  Type Protocol RT2          Se0/0       *HDLC*          Up     250       IS   ES-IS RT1#  show clns interface serial0/0  Serial0/0 is up, line protocol is up  Checksums enabled, MTU 1500, Encapsulation HDL  C   ERPDUs enabled, min. interval 10 msec., last sent 14:13:29   RDPDUs enabled, min. interval 100 msec., Addr Mask enabled   Congestion Experienced bit set at 4 packets   CLNS fast switching enabled   CLNS SSE switching disabled   DEC compatibility mode OFF for this interface   Next ESH/ISH in 28 seconds   Routing Protocol: IS-IS     Circuit Type: level-1-2     Interface number 0x1, local circuit ID 0x100     Level-1 Metric: 10, Priority: 64, Circuit ID: RT1.00     Number of active level-1 adjacencies: 0     Level-2 Metric: 10, Priority: 64, Circuit ID: RT1.00     Number of active level-2 adjacencies: 0     Next IS-IS Hello in 6 seconds 

Example 10-35 Debugging an MTU Mismatch on RT2

 RT2(config)#  interface s 0/0  RT2(config-if)#  mtu 2000  RT2(config-if)#  ^Z  RT2#  debug isis adj-packet  IS-IS Adjacency related packets debugging is on RT2#  Mar 10 20:44:16: ISIS-Adj: Sending serial IIH on Serial0/0, length 1999  Mar 10 20:44:21: ISIS-Adj: Rec serial IIH from *HDLC* (Serial0/0), cir type L1L9 Mar 10 20:44:21: ISIS-Adj: rcvd state DOWN, old state UP, new state INIT Mar 10 20:44:21: ISIS-Adj: Action = GOING DOWN Mar 10 20:44:21: %CLNS-5-ADJCHANGE: ISIS: Adjacency to RT1 (Serial0/0) Down, nes Mar 10 20:44:21: ISIS-Adj: L2 adj count 0 Mar 10 20:44:21: ISIS-Adj: Sending serial IIH on Serial0/0, length 1999 Mar 10 20:44:29: ISIS-Adj: Rec serial IIH from *HDLC* (Serial0/0), cir type L1L9 Mar 10 20:44:29: ISIS-Adj: rcvd state DOWN, old state DOWN, new state INIT Mar 10 20:44:29: ISIS-Adj: Action = GOING UP, new type = L2 Mar 10 20:44:29: ISIS-Adj: New serial adjacency Mar 10 20:44:29: ISIS-Adj: Sending serial IIH on Serial0/0, length 1999 Mar 10 20:44:38: ISIS-Adj: Rec serial IIH from *HDLC* (Serial0/0), cir type L1L9 Mar 10 20:44:38: ISIS-Adj: rcvd state DOWN, old state INIT, new state INIT Mar 10 20:44:38: ISIS-Adj: Action = GOING UP, new type = L2 Mar 10 20:44:38: ISIS-Adj: Sending serial IIH on Serial0/0, length 1999 Mar 10 20:44:47: ISIS-Adj: Sending serial IIH on Serial0/0, length 1999 RT2#  show clns neighbors  System Id    Interface   SNPA           State  Holdtime  Type Protocol RT1          Se0/0       *HDLC*         Init   27        L2   IS-IS RT2#  show clns interface  Serial0/0 is up, line protocol is up   Checksums enabled, MTU 2000, Encapsulation HDLC   ERPDUs enabled, min. interval 10 msec., last sent 12:52:30   RDPDUs enabled, min. interval 100 msec., Addr Mask enabled   Congestion Experienced bit set at 4 packets   CLNS fast switching enabled   CLNS SSE switching disabled   DEC compatibility mode OFF for this interface   Next ESH/ISH in 7 seconds   Routing Protocol: IS-IS     Circuit Type: level-1-2     Interface number 0x0, local circuit ID 0x100     Level-1 Metric: 10, Priority: 64, Circuit ID: RT2.00     Number of active level-1 adjacencies: 0     Level-2 Metric: 10, Priority: 64, Circuit ID: RT2.00     Number of active level-2 adjacencies: 0     Next IS-IS Hello in 7 seconds 

Some network operators hold the notion that because default data-link capabilities are mostly standardized, known up front, and configurable, padding hellos to ensure MTU consistency unnecessarily wastes precious network bandwidth. This has prompted the introduction of a command in IOS to turn off hello padding. Two commands are available to support this capability:

[no] hello padding [multi-point point-to-point] ” Can be applied globally at the router level

[no] isis hello padding ” For interface-level configuration

Example 10-36 illustrates the effect of disabling IS-IS hello padding on a serial interface and changing the MTU to 2000. The debug isis adj-packets command shows the size of hello packets to be only 38 bytes with padding of hellos disabled. Despite the higher MTU on one side on the link, the adjacency is retained.

Example 10-36 Disabling Hello Padding

 RT1(config-router)#  int se0/0  RT1(config-if)#  no isis hello padding  RT1(config-if)#  mtu 2000  RT1(config-if)#  ^Z  RT1#  show clns interface Serial0/0  Serial0/0 is up, line protocol is up  Checksums enabled, MTU 2000, Encapsulation HD  LC   ERPDUs enabled, min. interval 10 msec.   RDPDUs enabled, min. interval 100 msec., Addr Mask enabled   Congestion Experienced bit set at 4 packets   CLNS fast switching enabled   CLNS SSE switching disabled   DEC compatibility mode OFF for this interface   Next ESH/ISH in 40 seconds   Routing Protocol: IS-IS     Circuit Type: level-1-2     Interface number 0x1, local circuit ID 0x100     Level-1 Metric: 10, Priority: 64, Circuit ID: RT2.00     Number of active level-1 adjacencies: 0     Level-2 Metric: 10, Priority: 64, Circuit ID: 0000.0000.0000.00     Number of active level-2 adjacencies: 1     Next IS-IS Hello in 3 seconds     No hello padding RT1#  debug isis adj-packets  *Mar  1 03:41:57: ISIS-Adj: Rec serial IIH from *HDLC* (Serial0/0), cir type L19 *Mar  1 03:41:57: ISIS-Adj: rcvd state UP, old state UP, new state UP *Mar  1 03:41:57: ISIS-Adj: Action = ACCEPT *Mar  1 03:41:58: ISIS-Adj: Sending serial IIH on Serial0/0, length 38 *Mar  1 03:42:06: ISIS-Adj: Rec serial IIH from *HDLC* (Serial0/0), cir type L19 *Mar  1 03:42:06: ISIS-Adj: rcvd state UP, old state UP, new state UP *Mar  1 03:42:06: ISIS-Adj: Action = ACCEPT *Mar  1 03:42:06: ISIS-Adj: Sending serial IIH on Serial0/0, length 38 RT1#  show clns neighbors  System Id    Interface   SNPA           State  Holdtime  Type Protocol RT2          Se0/0       *HDLC*         Up     22        L2   IS-IS 

Misconfigured IP Addresses and Subnets

Originally, the IS-IS protocol relied on only IS-IS hellos to form and maintain adjacencies, using a three-way handshake on broadcast links and a two-way handshake on point-to-point links, all independent of the IP configuration. The increasing popularity of IS-IS for routing IP has promoted various enhancements both within the IETF and in vendor-specific implementations through feature introductions . One such feature introduced recently into Cisco IOS Software releases requires directly connected IP routers using IS-IS for routing to be also connected to the same IP subnet in order to form adjacencies.

This is not the case in older Cisco IOS Software releases, where directly connected routers could still form IS-IS adjacencies even though they were not properly configured to be on the same IP subnet. Recent changes in IOS require the source IP address of an IP neighbor to be validated before bringing up the adjacency. This behavior is implemented in Cisco IOS Software 12.0S releases, which are optimized for IP networks. This makes verification of the IP address configuration an important step in troubleshooting IS-IS adjacency problems.

The effect of IP address misconfiguration is illustrated in the following debugging and show command output (based on Figure 10-7). In Example 10-37, the IP address of RT2's serial interface is changed to 192.168.5.2/30. This erroneous entry causes adjacency to be invalidated at RT1 and logging of an adjacency change message. In this case, the IS-IS adjacency drops and it is replaced by an ES-IS adjacency. The debugging output shown in Example 10-32 includes an error, "ISIS-Adj: No usable IP interface addresses in serial IIH from 0." IP unnumbered configurations are not affected by the requirement that adjacent neighbors must be on the same subnet.

Figure 10-7. Test network for investigating IP address and subnet misconfiguration.

Example 10-37 Simulating and Debugging Misconfigured IP Addresses

 RT2#  config terminal  Enter configuration commands, one per line.  End with CNTL/Z. RT2(config)#  int s0/0  RT2(config-if)#  ip address 192.168.5.2 255.255.255.252  RT2(config-if)#  ^Z  RT2#  show clns neighbors  System Id   Interface   SNPA             State  Holdtime  Type Protocol RT1         Se0/0       *HDLC*           Up     257       IS   ES-IS RT6         Et0/0       00d0.58f7.8041   Up     7         L1   IS-IS RT1#  show clns neighbors  System Id    Interface   SNPA            State  Holdtime  Type Protocol RT2          Se0/0       *HDLC*          Up     284       IS   ES-IS RT5          Et0/0       00d0.58eb.ff01  Up     26        L1   IS-IS RT1#  debug isis adj-packets  Mar 10 21:44:19: ISIS-Adj: Sending serial IIH on Serial0/0, length 1499 Mar 10 21:44:20: ISIS-Adj: Rec serial IIH from *HDLC* (Serial0/0), cir type L1L9  Mar 10 21:44:20: ISIS-Adj: No usable IP interface addresses in serial IIH from  0 Mar 10 21:44:24: %CLNS-5-ADJCHANGE: ISIS: Adjacency to RT2 (Serial0/0) Down, hod Mar 10 21:44:24: ISIS-Adj: L2 adj count 0 Mar 10 21:44:27: ISIS-Adj: Sending serial IIH on Serial0/0, length 1499 Mar 10 21:44:30: ISIS-Adj: Rec serial IIH from *HDLC* (Serial0/0), cir type L1L9 Mar 10 21:44:30: ISIS-Adj: No usable IP interface addresses in serial IIH from 0 

IS-IS Route Maintenance Problems

This section focuses on route maintenance problems. The adjacency formation problems discussed in the preceding section are much easier to troubleshoot and resolve than router maintenance problems, which are normally not obvious in very large networks until a specific address or subnet becomes unreachable. Obviously, most IS-IS problems relate to adjacency problems. When no adjacency issues exist, it becomes challenging to isolate routing problems; however, on Cisco routers, the routing table might be fed with routing information from multiple sources. Most routers connected to the Internet are configured with two IP routing protocols, typically BGP for interdomain routing and IS-IS or OSPF for intradomain routing. Frequently in these situations, little overlap occurs in prefixes advertised by each protocol, so there is practically no contention in populating the routing table.

The routes in the IP routing table are organized into more efficient data structures for faster route lookup. Fast packet forwarding can be achieved on Cisco routers with two basic mechanisms:

• Fast switching

• Cisco Express Forwarding

Overviews of these mechanisms are provided in Chapter 1, "Overview of IP Routing." Refer to the references provided at the end of this chapter for further details about these mechanisms. Even though very rare, route maintenance problems might relate to problems in the lookup mechanism rather than the actual source of the route. This section focuses on only IS-IS-related causes. The following are the most common causes of routing inconsistencies in IS-IS and are discussed in detail in the following sections:

• IS-IS route advertisement problems

• IS-IS route installation problem

• Discontiguous Level 2 subdomain

• Route flaps

• LSP corruption storms

• Authentication problems

• IS-IS summarization and redistribution problems

IS-IS Route Advertisement Problems

Route advertisement problems refer to the inability of accurate routing information to reach a remote destination from a specific router or section of the network. As a link-state protocol, IS-IS routing information is advertised by means of link-state packets. Troubleshooting routing basically involves inspecting the LSP of the source node at both the source and routers in the region of the network where the routes are missing.

An IS-IS router advertises routing information to the rest of the network by one of the following methods :

• The route is a subnet connected to the router, and the corresponding interface is enabled for IS-IS routing.

• Passive interface.

• External routes by means of redistribution from another routing source (static, another dynamic routing protocol such as RIP, or a connected interface).

• Route summaries.

If a route is not being heard in the rest of the area or domain, the first step in troubleshooting the problem is to make sure the procedure for introducing the route into the IS-IS process has been properly configured.

Advertising IP Subnets by Activating IS-IS on Interfaces

If the first method is used, the ip router isis command must be applied to the appropriate interface. Note that IS-IS does not use a network statement for advertising an IP route as is commonly done for other protocols. Instead, enabling IS-IS on an interface triggers the formation of adjacencies on that interface and also advertises the attached IP subnet in an LSP to all neighbors. The show ip interface brief command confirms the interface on which the route is connected, followed by a show clns interface command. The actual configuration of the interface can also be verified with the command show running-config begin interface <type and number>, as shown in Example 10-38.

Example 10-38 Advertising IS-IS Routing Information

 RT1#  show run     begin interface Serial0/0  interface Serial0/0  mtu 2000  ip address 192.168.1.1 255.255.255.252  no ip directed-broadcast  ip router isis  no ip mroute-cache  no fair-queue  clockrate 2000000  no isis hello padding 

If the interface is properly configured, the next step might be to take a look at the IP reachability information fields of the router's LSP with the command show isis database level detail LSPID. This command provides insight into the information in the LSP that is advertised to other neighbors. It is assumed that there are no adjacency problems with any of the neighbors in the direction of the network where the route is missing. The level-1 keyword is used if the route is missing in only the local area, and the level-2 keyword is used if the route is not present in other areas within the same domain. If no adjacency problems exist, IS-IS routing is enabled correctly on the interface where the route should be taken from, and the prefix is seen in the LSP of the local router; then the problem is complex and requires more insight. The show isis database level detail LSPID command should be used on the remote routers to check the presence and currency of the LSP in question. The debug isis update-packet command assists with debugging any issues with periodic database synchronization on LANs. Note that synchronization issues are absent on point-to-point links because of the reliable flooding process used on such links.

Advertising a Subnet with passive-interface

The passive-interface command is normally used when the subnet on an interface needs to be advertised without forming adjacency or sending redundant hello messages over that interface. For example, a loopback interface is normally defined as a passive interface so that its address will be advertised without wasting CPU cycles to generate unnecessary hellos to nonexistent neighbors. If a loopback address is not advertised, the configuration should be checked to make sure it is specified as a passive interface. Current Cisco IOS Software releases remove the ip router isis command from the interface configuration when the interface is defined as passive.

Advertising External Routes

The most complicated routing maintenance problems involve redistribution. Redistribution of static routes is manageable and related problems are easy to troubleshoot.

Problems that relate to redistribution of a dynamically learned route are far more complicated. This is primarily because no inherent loop-prevention mechanisms are associated with route redistribution. The safest way to redistribute routes without establishing loops and route feedback is to limit the points where redistribution occurs. When problems occur in such situations, it is best to troubleshoot at the points of redistribution (border routers). Redistributed IP routes are entered as externals into the LSP. The show isis database level detail LSPID command can be used to inspect the LSPs of the border routers to ensure the external reachability information reaches the LSPs correctly. When the IS-IS domain has multiple points of redistribution, more than one LSP can introduce the external routes into the IS-IS domain, requiring care to be taken so that the border routers do not point to each other as the best exit from the domain to the external destinations.

The router-level display-route-detail command provides knowledge about which LSPs are responsible for specific routes in the routing table. This is a useful tool for troubleshooting routing problems and tracking where the entries in the routing table originate from. Figure 10-8 demonstrates the operation of this command, and the command output is shown in Example 10-39. A partial configuration of RT1 and the output of the show ip route isis command executed on RT1 are included in Example 10-39. Also shown is a show isis database command from the same router. Each IS-IS entry in the routing table of RT1 indicates the number of the LSP from which it came. The LSP numbers are shown in brackets in the show isis database output. For example, 10.1.2.0/24 is learned from LSP number 4, which is RT2.00-00. The display-route-detail command also displays backup paths for routes, which are alternative, less-preferred paths. The following backup entry is provided for this route:

Figure 10-8. Demonstrating use of the display-route-detail command.

 Backup ix/lvl/metric:9/L2/30 

The interpretation of this backup path is as follows :

• ix ” Index of source LSP (LSP 9)

• lvl ” Level 1 or Level 2 (Level 2)

• metric ” The metric value (30)

This extra information provided by the display-route-detail command gives more insight into the routing environment and can significantly help you to troubleshoot routing problems. In this case, 10.1.2.0/24 is learned from LSP4 (RT2.00-00) as primary, but a backup path is being advertised through LSP9 (RT4.00-00). The origins of these LSPs are obvious from the names .

Example 10-39 display-route-detail Command Example

 RT1#  show running-config  [snip] router isis  passive-interface Loopback0  default-information originate  net 49.0001.0000.0000.0001.00  metric-style wide  log-adjacency-changes  display-route-detai  l RT1#  show ip route isis  10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks  i L2 10.1.2.0/24 [115/20] via 192.168.1.2, Serial0/0, from LSP  4  Backup ix/lvl/metric:  9/L2/3  0      11.0.0.0/32 is subnetted, 6 subnets i L1    11.1.1.3 [115/10] via 10.1.1.3, Ethernet0/0, from LSP 7         Backup ix/lvl/metric:  8/L2/10 i L2    11.1.1.2 [115/10] via 192.168.1.2, Serial0/0, from LSP 4         Backup ix/lvl/metric:  9/L2/30 i L2    11.1.1.6 [115/20] via 192.168.1.2, Serial0/0, from LSP 4         Backup ix/lvl/metric:  9/L2/30 i L1    11.1.1.5 [115/10] via 10.1.1.5, Ethernet0/0, from LSP 5         Backup ix/lvl/metric:  8/L2/20 i L2    11.1.1.4 [115/20] via 192.168.1.2, Serial0/0, from LSP 9         Backup ix/lvl/metric:  4/L2/20      192.168.2.0/30 is subnetted, 1 subnets i L1    192.168.2.0 [115/20] via 10.1.1.3, Ethernet0/0, from LSP 7         Backup ix/lvl/metric:  4/L2/30  8/L2/110  */L2/120 RT1#  show isis data  IS-IS Level-1 Link State Database LSPID           LSP Seq Num  LSP Checksum  LSP Holdtime      ATT/P/OL RT1.00-00     * 0x000000FA   0xAF5F        661               1/0/0  (1) RT1.01-00     * 0x000000F4   0x40FC        349               0/0/0  (3) RT3.00-00       0x0000119B   0xCA8D        668               1/0/0  (7) RT5.00-00       0x000000EC   0x1E90        434               0/0/0  (5) IS-IS Level-2 Link State Database LSPID           LSP Seq Num  LSP Checksum  LSP Holdtime      ATT/P/OL RT1.00-00    * 0x00000101   0x9FB7        563               0/0/0  (2) RT1.01-00    * 0x0000000D   0x8D30        1193              0/0/0  (6)  RT2.00-00      0x00001FC0   0x1F57        739               0/0/0  (4  ) RT2.01-00      0x00000001   0xBA0C        730               0/0/0  (10) RT3.00-00      0x0000119D   0x01F4        714               0/0/0  (8)  RT4.00-00      0x00001191   0x06F3        834               0/0/0  (9)  

Advertising Summary Routes

Frequently, network operators attempt to summarize routing information by representing a set of routes with one or a fewer number of prefixes than in the original set. This is good practice and helps save on system resources, such as memory and network bandwidth. If many smaller prefixes are summarized into one prefix, less space is required in the LSP to advertise those prefixes, and, therefore, less bandwidth is needed to flood the smaller LSP throughout the area. When routes are summarized, they might become part of an aggregate. Therefore, troubleshooting missing routes in such environments should involve checking whether they are represented accurately by the summaries as intended.

IS-IS Route Installation Problems

Route installation problems involve situations where an LSP from a remote router is properly received, but a route in the LSP is not installed in the routing table as expected. IS-IS does not have any complicated schemes for installing routes that are determined by the SPF process to be the best path. For example, OSPF external link-state advertisements have a forwarding address when non-zero has to be learned through OSPF for a routing bit that controls insertion into the routing table to be set appropriately. IS-IS has no concept of a routing bit. The most plausible reason why an IS-IS route should not make it into the routing table is because there is a similar route from another routing source with a better administrative distance than IS-IS. Route installation problems are rare in IS-IS, and when they do occur, the reason is most certainly a software failure or an interoperability issue.

You can use the following commands to isolate route installation problems:

• show ip route [isis]

• show isis database level detail LSPID

The show ip route prefix command determines the absolute absence of the prefix from the routing table. If the route is present but from another source, the administrative distance should be lower; otherwise , there's an issue. The isis keyword lists only all IS-IS routes in the IP routing table. This might prove useful to confirm whether only a specific route from IS-IS is affected.

If a route is not in the routing table as expected, the show isis database command specifying the routing level, the LSPID of the source LSP, and the detail keyword should be used to probe further into the contents of the LSP. Help from an expert might be required to confirm the problem.

Unstable IS-IS Routes

Route instability means that the route is available only intermittently. This is usually described as a route flap. Route flap might result just from an unstable link or possibly because of a more complex underlying condition, such as an intermittent routing loop.

Typically, at the point where the flap is seen, the LSP that contains the route is periodically advertised and withdrawn, or newer versions are continuously being received. Route flaps can also have a devastating effect on the routing environment if a large number of LSPs and routers are affected. This might cause the SPF process to run for prolonged periods, resulting in potentially dangerous levels of CPU utilization on the affected routers.

If the network changes affect only IP prefixes, only partial route calculation (PRC) might be performed by the IS-IS SPF process. Other situations might require scheduling of a "full" SPF. The latter is more CPU- intensive .

By far, the most common causes of route flaps are unstable links. Another well-known cause is corruption of an LSP in a network device as it is being flooded through the network. LSP corruption storms are discussed later in this chapter.

In a large network, the display-route-detail command discussed in the preceding section can determine the LSP associated with the unstable route (refer to Example 10-35). The focus of problem isolation can then be placed on the source of the LSP.

In most cases, a route flap problem can be quickly confirmed by looking at the sequence numbers of LSPs in the same link state database. The show isis database output in Example 10-40 features a far higher sequence number for the LSP with ID RT2.00-00 than for the other known LSPs. The huge discrepancy signals either problems at the source or somewhere in between the source and the point of observation. If the source and the router at which the problem is being observed are directly connected, you can use standard procedures to troubleshoot the physical and data link layers . The show interface or show clns interface command can provide link status information and some leads might be available in the logs. Because the problem might be adjacency related, the debug isis adj-packet debugging option can be enabled to observe the problem further. At the source of the LSP, the debug isis spf-log command provides information regarding events affecting the locally sourced LSP and, therefore, some clues to the problem (refer to Example 10-12 and Table 10-2).

Example 10-40 Troubleshooting Unstable Routes

 RT1#  show isis database  IS-IS Level-1 Link State Database LSPID             LSP Seq Num  LSP Checksum  LSP Holdtime      ATT/P/OL RT1.00-00       * 0x00000093   0x7EF7        438               1/0/0 RT1.01-00       * 0x0000008B   0xA014        838               0/0/0 RT5.00-00         0x00000085   0xEC29        1107              0/0/0 IS-IS Level-2 Link State Database LSPID             LSP Seq Num  LSP Checksum  LSP Holdtime      ATT/P/OL RT1.00-00       * 0x00000095   0xA819        1082              0/0/0  RT2.00-00         0x00001F57   0xE0FE        781               0/0/0  

LSP Corruption Storms

LSP corruption storms occur when LSPs are corrupted by a network device as the LSPs are flooded. When a router receives a corrupted LSP, it purges it from the network by setting its remaining lifetime to 0 and flooding it back into the network. Corrupted LSPs are determined through checksum matching. When a corrupted LSP is purged, the source regenerates another copy and re-advertises it back into the network. This advertise-and-purge activity will likely occur until the source of the corruption is removed from the network. Corruption storms consume network resources, such as CPU and bandwidth, and can seriously impact network performance. LSP corruption storms can be eliminated with the router-level command ignore-lsp-errors.

Discontiguous Level 2 Subdomains

IS-IS requires the Level 2 backbone that interconnects the various areas to be contiguous. This condition can easily be violated by bad network design or router misconfiguration, as shown in Figure 10-9. Cisco routers function as Level 1-2 by default and caution should be exercised in turning off Level 2 routing until the impact is well understood . The Level 1-only router in area 49.0002 disrupts the continuity of the Level 2 path, preventing areas 49.0001 and 49.0003 from reaching each other.

Figure 10-9. Case study for fragmented Level 2 backbone.

Authentication Problems

IS-IS specifications (ISO 10589 [1] and RFC 1195 [2]) provide a simple password scheme for authentication. Three kinds of authentication methods that use simple password are supported on Cisco routers, as discussed in Chapter 9.

• Link authentication

• Area authentication

• Domain authentication

When authentication is configured, a clear-text password is inserted into IS-IS packets, such as LSPs, CSNPs, and PSNPs, as are also hellos (link authentication only). The password is verified before a packet is accepted for processing. Clearly, a password mismatch between the source of the packet and the recipient could create both adjacency and routing maintenance problems, depending on the type of authentication enabled. Example 10-41 shows a debug adj-packet output for a link authentication failure.

Example 10-41 Debugging Authentication Failures

 RT1#  debug isis adj-packets  *Apr 23 04:25:36: ISIS-Adj: Rec serial IIH from *HDLC* (Serial0/0), cir type L1L2, cir id 00, length 1499 *Apr 23 04:25:36: ISIS-Adj: Authentication failed *Apr 23 04:25:42: ISIS-Adj: Sending serial IIH on Serial0/0, length 1499 *Apr 23 04:25:46: ISIS-Adj: Rec serial IIH from *HDLC* (Serial0/0), cir type L1L2, cir id 00, length 1499 *Apr 23 04:25:46: ISIS-Adj: Authentication failed *Apr 23 04:25:50: ISIS-Adj: Sending serial IIH on Serial0/0, length 1499