MLS
Multilayer Switching (MLS) is Cisco's Ethernet-based routing switch technology. MLS is currently supported in two platforms: the Catalyst 5000 and the Catalyst 6000. The Catalyst 5000 makes use of the NetFlow Feature Card (NFFC) I or II to provide hardware-assisted routing. The Catalyst 6000 performs the same operations using the Multilayer Switch Feature Card (MSFC) in conjunction with the Policy Feature Card (PFC). In keeping with the chronological presentation of this chapter, this section focuses on the Catalyst 5000's implementation of MLS. The Catalyst 6000's Layer 3 capabilities are discussed in the "Catalyst 6000 Layer 3 Switching" section later in the chapter and in Chapter 18, "Layer 3 Switching and the Catalyst 6000/6500s." Also, although MLS supports both IP and IPX traffic, this section focuses on IP.
Note
IPX MLS is supported on all Catalyst 6000s using a Multilayer Switch Feature Card (MSFC, discussed later in the chapter). IPX MLS is supported on Catalyst 5000s using a NFFC II and 5.1+ software.
In its most basic sense, the NFFC is a pattern-matching engine. This allows the Catalyst to recognize a wide variety of different packets. By matching on various combinations of addresses and port numbers, the routing switch form of Layer 3 switching can be performed. However, a host of other features are also possible. By matching on Layer 3 protocol type, a feature called Protocol Filtering can be implemented. By matching on Internet Group Management Protocol (IGMP) packets, the Catalyst can perform IGMP Snooping to dynamically build efficient multicast forwarding tables. Finally, by matching on Layer 2 and Layer 3 QoS and COS information, traffic classification and differentiation can be performed.
This section initially only considers the Layer 3 switching aspects of the NFFC. The other capabilities are addressed at the end of the section (as well as in other chapters such as Chapter 13, "Multicast and Broadcast Services").
One of the important things to keep in mind when discussing MLS is that, like all shortcut switching mechanisms, it is a caching technique. The NFFC does not run any routingprotocols such as OSPF, EIGRP, or BGP.
It is also important to realize that MLS, formerly known as NetFlow LAN Switching, is a completely different mechanism than the NetFlow switching on Cisco's software-based routers. In its current implementation, NetFlow on the routers is targeted as a powerful data collection tool via NetFlow Data Export (although it can also be used to reduce the overhead associated with things like complex access lists). Although MLS also supports NetFlow Data Export (NDE), its primary mission is something very different Layer 3 switching.
Because the NFFC does not run any routing protocols, it must rely on its pattern-matching capabilities to discover packets that have been sent to a router (notice this is the device running protocols such as OSPF) and then sent back to the same Catalyst. It then allows the NFFC to shortcut future packets in a manner that bypasses the router. In effect, the NFFC notices that it sent a particular packet to the router, only to have the router send it right back. It then says to itself, "Boy, that was a waste of time!" and starts shortcutting all remaining packets following this same path (or flow).
Note
NetFlow defines a flow as being unidirectional. Therefore, when two nodes communicate via a bi-directional protocol such as Telnet, two flows are created.
Although MLS is fundamentally a very simple technique, there are many details involved. The following section presents an in-depth account of the entire MLS mechanism. Later sections examine how to configure and use MLS.
Detailed MLS Theory of Operations
MLS makes use of three components:
MLS Route Processor (MLS-RP)
MLS Switching Engine (MLS-SE)
Multilayer Switching Protocol (MLSP)
The MLS-RP acts as the router in the network (note that more than one can be used). This device handles the first packet in every flow, allowing the MLS-SE to build shortcut entries in a Layer 3 CAM table. The MLSP is a lightweight protocol used by the MLS-RP to initialize the MLS-SE and notify it of changes in the Layer 3 topology or security requirements. For simplicity, this chapter usually refers to the MLS-RP as the router and the MLS-SE as the NFFC.
MLS uses a four-step process:
MLSP hello packets are sent by the router
The NFFC identifies candidate packets
The NFFC identifies enable packets
The NFFC shortcuts future packets
The following sections describe each of these steps using the sample network shown in Figure 11-4.
Figure 11-4. Sample MLS Network
This network consists of two VLANs, VLAN 1 (Red) and VLAN 2 (Blue). Two end stations have been shown. Host-A has been assigned to the Red VLAN, and Host-B has been assigned to the Blue VLAN. An ISL-attached router has also been included. Its single Fast Ethernet interface (Fast Ethernet1/0) has been logically partitioned into two subinterfaces, one per VLAN. The IP and MAC addresses for all devices and subinterfaces are shown.
Figure 11-4 portrays the router as an ISL-attached external device using the router-on-a-stick configuration. Other possibilities include an RSM or a one-interface-per-VLAN attached router.
Step 1: MLSP Hello Packets Are Sent by the Router
When the router first boots, it begins sending MLSP hello packets every 15 seconds. These packets contain information on the VLANs and MAC addresses in use on the router. By listening for these hello packets, the NFFC can learn the attributes of any MLS-capable routers in the Layer 2 network. The NFFC associates a single XTAG value with every MLS router that it identifies. Because the MLSP hellos are periodic in nature, they allow routers and Catalysts to boot at random times while also serving as a router keepalive mechanism for the NFFC (if a router goes offline, its cache entries are purged).
Tip
There is one XTAG per MLS-capable router. The XTAG serves as a single handle for a router's multiple MAC addresses (each interface/VLAN could be using a different MAC address). XTAGs are locally significant (different NFFCs can refer to the same router with different XTAGs).
Figure 11-5 illustrates the MLS hello process.
Figure 11-5. MLS Hello Process
As shown in Figure 11-5, the MLSP packets are sourced from subinterface Fast Ethernet1/0.1 on the router (this is a configurable option; the router commands are presented later). These packets are then used to populate the Layer 2 CAM table (a form of bridging table commonly used in modern switches) with special entries that are used to identify packets going to or coming from a router interface (the show cam Catalyst command places an R next to these entries). Each router is also assigned a unique XTAG value. If a second router were present in Figure 11-5, it would receive a different XTAG number than the value of 1 assigned to the first router. However, notice that all MAC addresses and VLANs for a single router are associated with a single XTAG value.
Although it is not illustrated in Figure 11-5, the MLSP hello packets flow throughout the Layer 2 network. Because they are sent using a multicast address (01-00-0C-DD-DD-DD, the same address used by CGMP), non-MLS-aware switches simply flood the hello packets to every segment in VLAN 1. In this way, all MLS switches learn about all MLS-capable routers.
Step 2: The NFFC Identifies Candidate Packets
After Step 1 has allowed the NFFC to acquire the addresses of the MLS-capable routers, the NFFC starts using its pattern-matching capabilities to look for packets that are destined to these addresses. If a packet is headed to the router and does not have an existing shortcut entry (if it did have a shortcut entry, it would skip this step and be shortcut switched), it is classified as a candidate packet. The packet uses the normal Catalyst Layer 2 forwarding process and gets forwarded out the port connected to the router.
Note
Candidate packets must meet the following criteria:
They have a destination address equal to one of the router MAC addresses learned via MLSP.
They do not have an existing shortcut entry.
For example, refer to Figure 11-6 and assume that Host-A Telnets to Host-B. Recognizing that Host-B is in a different subnet, Host-A sends the packets to its default gateway, subinterface 1/0.1 on the router.
Figure 11-6. A Candidate Packet
Figure 11-7 illustrates the relevant fields in this packet as it traverses the ISL link to the router.
Figure 11-7. Candidate Packet Fields
The ISL header contains a VLAN ID of 1. The Ethernet header contains a source MAC address equal to Host-A and a destination MAC address equal to 00-00-0C-11-11-11, the MAC address of subinterface 1/0.1 on the router. The source and destination IP addresses belong to Host-A and Host-B, respectively. The switch uses the destination MAC address to perform two actions:
It forwards the packet out Port 1/1 toward the router using Layer 2 switching.
It recognizes the MAC address destination address as one of the router's addresses learned in Step 1. This triggers a lookup for an existing Layer 3 shortcut entry based on the destination IP address (other options are available, but these are discussed later). Assuming that a shortcut does not exist (it is a new flow), the packet is flagged as a candidate packet and a partial shortcut entry is created.
Step 3: The NFFC Identifies Enable Packets
The router receives and routes the packet as normal. Recognizing the destination address as being directly connected on subinterface Fast Ethernet1/0.2, the router sends the packet back across the ISL link encapsulated in VLAN 2 as illustrated in Figure 11-8.
Figure 11-8. An Enable Packet
Figure 11-9 shows the relevant fields contained in the packet as it crosses the ISL link between the router and switch.
Figure 11-9. Enable Packet Fields
The router has rewritten the Layer 2 header. Not only has it changed the VLAN number in the ISL header, it has modified both MAC addresses. The source MAC address is now equal to 00-00-0C-22-22-22, the MAC address used on the router's Fast Ethernet1/0.2 subinterface, and the destination address is set to Host-B. Although the IP addresses have not been changed, the router must modify the IP header by decrementing the Time To Live (TTL) field and update the IP checksum.
As the packet traverses the Catalyst on its way from the router to Host-B, five functions are performed:
The destination MAC address is used to Layer 2 switch the packet out Port 3/1.
The NFFC recognizes the source MAC address as one of the entries created in Step 1 via the hello process.
The NFFC uses the destination IP address to look up the existing partial shortcut entry created in Step 2.
The NFFC compares the XTAG values associated with the source MAC address of this packet and the partial shortcut entry. Because they match, the NFFC knows that this is the enable packet coming from the same router targeted by the candidate packet.
The NFFC completes the shortcut entry. This entry will contain all of the information necessary to rewrite the header of future packets (in other words, the fields shown in Figure 11-9).
Step 4: The NFFC Shortcuts Future Packets
As future packets are sent by Host-A, the NFFC uses the destination IP address to look up the completed shortcut entry created in Step 3. Finding a match, it uses a rewrite engine to modify the necessary header information and then sends the packet directly to Host-B (the packet is not forwarded to the router). The rewrite operation modifies all of the same fields initially modified by the router for the first packet. From Host-B's perspective, it has no idea that the NFFC has intercepted the packet. Figure 11-10 illustrates this operation.
Figure 11-10. A Shortcut Packet
The rewrite mechanism can modify the following fields:
Source and Destination MAC address
VLAN ID
TTL
Encapsulation (for example, ARPA to SNAP)
Checksums
ToS/COS
Note
It is important to understand that, although MLS is a Cisco-specific feature, it is entirely standards compliant. Unlike some other shortcut and cut-through mechanisms, MLS makes all of the modifications that a normal router makes to an IP and IPX packet. In fact, if you were to use a protocol analyzer to capture traffic going through MLS and a normal router, you would not be able to tell the difference.
There are two options that MLS can use to rewrite the packet. In the first option, the NFFC card itself is used to rewrite the packet. The NFFC actually contains three rewrite engines, one per Catalyst 5500 bus. These rewrite engines are referred to as central rewrite engines. The downside of using a central rewrite engine is that it requires the packet to traverse the bus twice. For example, in Figure 11-10, the packet first arrives through Port 2/1 and is flooded across the backplane as a VLAN 1 frame. The NFFC is treated as the destination output port. After the NFFC has completed the shortcut lookup operation, it uses the rewrite information contained in the Layer 3 CAM table to update the packet appropriately. It then sends the rewritten packet back across the bus as a VLAN 2 frame, where the Layer 2 CAM table is used to forward it out Port 3/1. In other words, it crosses the bus first as a packet in the Red VLAN and again as a packet in the Blue VLAN. As a result, performance is limited to approximately 750,000 pps (on Catalyst 5000s).
The second rewrite option uses a feature called inline rewrite to optimize this flow. When using Catalyst modules that support this feature, the rewrite operation can be performed on the output module itself, allowing the packet to cross the bus a single time. Figure 11-11 illustrates the inline rewrite operation.
Figure 11-11. Inline Rewrite
When the packet comes in from Host-A, it is flooded across the bus. All ports make a copy of the frame, including the destination Port 3/1 and the NFFC. The NFFC looks up the existing shortcut entry and sends just the rewrite information to Module 3 (this occurs on a separate bus from the data bus). Module 3 uses its local rewrite engine to modify the packet and immediately forwards it out Port 3/1. Because the frame only traversed the bus once, throughput is doubled to approximately 1,500,000 pps.
Note
The central rewrite versus inline rewrite issue is not a problem on the Calayst 6000 because all of its Ethernet line cards support inline rewrite.
Cache Aging
To prevent the MLS cache from overflowing, an aging process must be run. This is a software-controlled operation that runs in the background. Although the architecture of current NFFCs can theoretically hold 128,000 entries, it is recommended to keep the total number of entries below 32,000 on current versions of the card. MLS supports three separate aging times:
Quick
Normal
Fast
Quick aging is utilized to age out partial shortcut entries that never get completed by an enable packet. The aging period for these entries is fixed at five seconds.
Normal aging is used for the typical sort of data transfer flow. This is a user-configurable interval that can range from 64 to 1,920 seconds with the set mls agingtime [agingtime] command. The default is 256 seconds. When changing the default value, it is rounded to the nearest multiple of 64 seconds.
Fast aging is used to age short-term data flows such as DNS, ping, and TFTP. The Fast aging time can be adjusted with the set mls agingtime fast [fastagingtime] [pkt_threshold] command. If the entry does not have more than pkt_thresholdpackets within fastagingtimeseconds, the entry is removed. By default, Fast aging is not enabled because the fastagingtimeme parameter is set to 0. The possible fastagingtime values are 0, 32, 64, 96, and 128 seconds (it uses the nearest value if you enter a different value). The pkt_threshold parameter can be set to 0, 1, 3, 7, 15, 31, or 63 (again, you can enter other values and it uses the closest possible value).
Access Lists and Flow Masks
One of the best features of MLS is that it supports access lists. Both standard and extended output IP access lists are available. This support relies on three mechanisms:
The assumption that if a candidate packet fails an access list, the router never sends an enable packet to complete the shortcut
The MLSP protocol to notify the NFFC to flush all shortcut entries if the access list is modified
A flow mask
The first mechanism handles the case where a packet is forwarded to the router and never returned to any Catalyst because it failed an access list. As a result, MLS can be a safe and effective technique.
The MLSP flush mechanism provides important integration between the router and the NFFC. If the router is configured with an access list, the MLSP protocol can be used to cause all cache entries to be flushed (forcing new entries to be processed by the access list). The flush mechanism is also used to remove cache entries after a routing table change.
The flow mask is used to set the granularity with which the NFFC determines what constitutes a flow. In all, three flow masks are possible:
Destination flow mask.
Destination-source flow mask
Full flow mask
A destination flow mask enables flows based on Layer 3 destination addresses only. A single shortcut is created and used for all packets headed to a specific destination IP (or IPX) address, regardless of the source node or application. This flow mask is used if no access lists are configured on the router.
A destination-source flow mask uses both the source and destination Layer 3 addresses. As a result, each pair of communicating nodes uses a unique shortcut entry. However, all of the applications flowing between each pair of nodes uses the same shortcut entry. This flow mask is used if a standard access list or a simple extended access list without port numbers is in use on the router.
A full flow mask uses Layer 4 port numbers in addition to source and destination Layer 3 addresses. This creates a separate shortcut for every application flowing between every pair of nodes. By doing so, a full flow mask provides the highest level of control and allows the NFFC to perform Layer 4 switching. Because it tracks flows at the application level, it can also be used to provide very detailed traffic statistics via NetFlow Data Export (NDE), a feature that is discussed in more detail later. The full flow mask is applied if extended access lists referencing port numbers are in use.
For example, consider the network shown in Figure 11-12.
Figure 11-12. A Sample MLS Network
Host-A and Host-B are assigned to VLAN 1, whereas Host-C is in VLAN 2. The Catalyst and the router have been correctly configured for MLS. Example 11-5 shows the output of the show mls entry command when using a destination flow mask.
Example 11-5 Sample show mls entry Output When Using a Destination Flow Mask
Cat-A> (enable) show mls entry Last Used Last Used Destination IP Source IP Prot DstPrt SrcPrt Destination Mac Vlan Port --------------- --------------- ---- ------ ------ ----------------- ---- ----- MLS-RP 10.1.1.1: 10.1.1.2 10.1.2.2 TCP 11000 Telnet 00-00-0c-7c-3c-90 1 2/16 10.1.1.3 10.1.2.2 ICMP - - 00-60-3e-26-96-00 1 2/15 10.1.2.2 10.1.1.3 ICMP - - 00-00-0c-5d-0b-f4 2 2/17 Because only three destination IP addresses exist between the two VLANs in this network, only three lines are displayed with a destination flow mask. Notice that all of the traffic flowing to a single destination address uses a single shortcut entry. Therefore, each line in the output only shows information on the most recent packet going to each destination. This fact is reflected in the column headers that use names such as Last Used Source IP.
Example 11-6 shows the same output after a destination-source flow mask has been configured.
Example 11-6 Sample show mls entry Output When Using a Destination-Source Flow Mask
Cat-A> (enable) show mls entry Last Used Destination IP Source IP Prot DstPrt SrcPrt Destination Mac Vlan Port --------------- --------------- ---- ------ ------ ----------------- ---- ----- MLS-RP 10.1.1.1: 10.1.1.3 10.1.2.2 ICMP - - 00-60-3e-26-96-00 1 2/15 10.1.2.2 10.1.1.3 ICMP - - 00-00-0c-5d-0b-f4 2 2/17 10.1.1.2 10.1.2.2 TCP 61954 Telnet 00-00-0c-7c-3c-90 1 2/16 10.1.2.2 10.1.1.2 TCP Telnet 61954 00-00-0c-5d-0b-f4 2 2/17 Example 11-6 displays two lines of output for every pair of nodes that communicate through the router (one for each direction). For example, the first two lines indicate that the last packets to travel between 10.1.1.3 and 10.1.2.2 was a ping (the first line shows the flow from 10.1.1.3 to 10.1.2.2, and the second line shows the flow in the opposite direction). The last two lines show the two sides of a Telnet session between 10.1.1.2 and 10.1.2.2 (notice how the source and destination port number numbers are swapped). Notice that traffic between 10.1.1.2 and 10.1.1.3 do not show up (this information is Layer 2 switched and does not use MLS). Also notice that the "Last Used" header only applies to the Prot (protocol), DstPrt (destination port), and SrcPrt (source port) fields. It no longer applies to the Source IP field because every new source addresses creates a new shortcut entry.
Finally, Example 11-7 displays sample output after configuring a full flow mask.
Example 11-7 Sample show mls entry Output When Using a Full Flow Mask
Cat-A> (enable) show mls entry Destination IP Source IP Prot DstPrt SrcPrt Destination Mac Vlan Port --------------- --------------- ---- ------ ------ ----------------- ---- ----- MLS-RP 10.0.1.1: 10.0.1.2 10.0.2.2 TCP 11778 69 00-00-0c-7c-3c-90 1 2/16 10.0.2.2 10.0.1.3 TCP 110 11004 00-00-0c-5d-0b-f4 2 2/17 10.0.2.2 10.0.1.2 TCP 69 11778 00-00-0c-5d-0b-f4 2 2/17 10.0.1.2 10.0.2.2 TCP 65026 SMTP 00-00-0c-7c-3c-90 1 2/16 10.0.1.3 10.0.2.2 TCP 11002 Telnet 00-60-3e-26-96-00 1 2/15 10.0.1.2 10.0.2.2 TCP 12290 110 00-00-0c-7c-3c-90 1 2/16 10.0.1.2 10.0.2.2 TCP 11266 WWW 00-00-0c-7c-3c-90 1 2/16 10.0.1.2 10.0.2.2 TCP 64514 FTP 00-00-0c-7c-3c-90 1 2/16 10.0.2.2 10.0.1.2 TCP FTP 64514 00-00-0c-5d-0b-f4 2 2/17 10.0.2.2 10.0.1.3 TCP 69 11005 00-00-0c-5d-0b-f4 2 2/17 10.0.2.2 10.0.1.2 TCP WWW 63490 00-00-0c-5d-0b-f4 2 2/17 10.0.2.2 10.0.1.3 TCP 9 11001 00-00-0c-5d-0b-f4 2 2/17 10.0.2.2 10.0.1.3 ICMP - - 00-00-0c-5d-0b-f4 2 2/17 10.0.1.2 10.0.2.2 TCP 62978 9 00-00-0c-7c-3c-90 1 2/16 10.0.1.2 10.0.2.2 TCP 64002 20 00-00-0c-7c-3c-90 1 2/16 10.0.2.2 10.0.1.2 TCP Telnet 62466 00-00-0c-5d-0b-f4 2 2/17 10.0.1.2 10.0.2.2 TCP 63490 WWW 00-00-0c-7c-3c-90 1 2/16 10.0.1.2 10.0.2.2 TCP 62466 Telnet 00-00-0c-7c-3c-90 1 2/16 10.0.2.2 10.0.1.3 TCP Telnet 11002 00-00-0c-5d-0b-f4 2 2/17 10.0.2.2 10.0.1.2 TCP WWW 11266 00-00-0c-5d-0b-f4 2 2/17 10.0.1.3 10.0.2.2 TCP 11004 110 00-60-3e-26-96-00 1 2/15 10.0.2.2 10.0.1.2 TCP SMTP 65026 00-00-0c-5d-0b-f4 2 2/17 10.0.1.3 10.0.2.2 TCP 11005 69 00-60-3e-26-96-00 1 2/15 10.0.2.2 10.0.1.2 TCP 110 12290 00-00-0c-5d-0b-f4 2 2/17 10.0.2.2 10.0.1.3 TCP WWW 11003 00-00-0c-5d-0b-f4 2 2/17 10.0.1.3 10.0.2.2 TCP 11003 WWW 00-60-3e-26-96-00 1 2/15 10.0.2.2 10.0.1.2 TCP 20 64002 00-00-0c-5d-0b-f4 2 2/17 10.0.1.3 10.0.2.2 ICMP - - 00-60-3e-26-96-00 1 2/15 10.0.1.3 10.0.2.2 TCP 11001 9 00-60-3e-26-96-00 1 2/15 10.0.2.2 10.0.1.2 TCP 9 62978 00-00-0c-5d-0b-f4 2 2/17 Notice that Example 11-7 includes every pair of communicating applications (both IP addresses and port numbers are considered). Also notice that none of the fields include a Last Used header because all of the individual flows are fully accounted for.
The multiple flow masks allow the NFFC to track information at a sufficient level of granularity to ensure that denied packets do not slip through using a pre-existing shortcut entry. However, to be truly secure, input access lists need to process every packet. As a result, configuring an input access on the router disables MLS on that interface. However, an optional parameter was introduced in 12.0 IOS images to allow input access lists at the expense of some security risk. To enable this feature, specify the input-acl parameter on the end of the mls rp ip global router command (Step 1 in the five-step router configuration process discussed later).
Tip
The mls rp ip input-acl command can be used to enable input access lists at the expense of some fairly minor security risks.
If multiple routers are in use with different flow masks, all MLS-capable Catalysts use the most granular (longest) flow mask. In other words, if there are two routers without access lists and a third router with a standard access list, the destination-source flow mask is used.
If you are not using access lists but you want to use a source-destination or full flow mask, you can use the set mls {flow destination | destination-source | full } command to set a minimum flow mask. For example, by forcing the flow mask to full, you can collect detailed traffic statistics (see the "Using MLS" section).
Configuring MLS
Although the theory behind MLS is somewhat involved, it is fairly easy to configure. To fully implement MLS, you must separately configure the router and the Catalyst Supervisor.
MLS Router Configuration
To configure a Cisco router for MLS, use the following five-step process:
Globally enable MLS on the router. To do this, use the mls rp ip command. This command must be entered from the global configuration mode (in other words, not on a particular interface or subinterface).
Configure a VLAN Trunking Protocol (VTP) domain for each interface using the mls rp vtp-domain domain_name command. This is an interface configuration command. For ISL interfaces, it can only be entered on the major interface (all of the subinterfaces inherit the same VTP domain name). The vtp-domain command should be entered prior to completing the remaining steps. However, if no VTP domain has been assigned to the switches in your network, this step can be skipped.
If a non-trunk interface is used on an external router, the mls rp vlan-id vlan_number command must be used to tell the router about VLAN assignments. This command must be entered before the remaining steps can be completed. On an ISL interface, this command is not required because the encap isl vlan_number command performs the same function. In the case of an RSM, this command is also not required because the RSM automatically receives VLAN information.
Enable each MLS interface using the mls rp ip command. This is an interface or subinterface configuration command.
Select one or more router interfaces to send MLSP packets using the mls rp management-interface command. Again, this is an interface or subinterface command. In general, you only enter it on a single interface. Choose an interface connected to a VLAN that reaches all of the MLS-capable Catalysts in your network. If the command is not entered at all, no MLSP packets are sent, preventing MLS from functioning.
Tip
If no VTP domain has been specified on the Catalysts (check this with show vtp domain), you do not need to set one on the router (in other words, the Null domain is used). If you use the mls rp ip or mls rp management-interface commands before specifying a VTP domain, the interface is automatically assigned to the Null domain. To change the domain name to something else, you need to remove all mls rp commands from that interface and start reconfiguring it from scratch (current versions automatically remove the mls rp commands when you enter no mls rp vtp-domain domain_name).
Example 11-8 illustrates a router MLS configuration that is appropriate for the example presented in Figures 11-4 through Figure 11-11.
Example 11-8 External Router MLS Configuration
mls rp ip ! interface FastEthernet1/0 mls rp vtp-domain Skinner ! interface FastEthernet1/0.1 ip address 10.1.1.1 255.255.255.0 mls rp management-interface mls rp ip ! interface FastEthernet1/0.2 ip address 10.1.2.1 255.255.255.0 mls rp ip
The corresponding RSM configuration appears as in Example 11-9.
Example 11-9 RSM Router MLS Configuration
mls rp ip ! interface Vlan1 ip address 10.1.1.1 255.255.255.0 mls rp vtp-domain Skinner mls rp management-interface mls rp ip ! interface Vlan2 ip address 10.1.2.1 255.255.255.0 mls rp vtp-domain Skinner mls rp ip
And finally, the same configuration running on a router using two Ethernet ports looks like Example 11-10.
Example 11-10 External Router MLS Configuration for Multiple Ethernet Ports
mls rp ip ! interface Ethernet0 ip address 10.1.1.1 255.255.255.0 mls rp vtp-domain Skinner mls rp vlan-id 1 mls rp management-interface mls rp ip ! interface Ethernet1 ip address 10.1.2.1 255.255.255.0 mls rp vtp-domain Skinner mls rp vlan-id 2 mls rp ip
MLS Switch Configuration
The switch configuration is considerably more straightforward. In fact, if you are using MLS in conjunction with an RSM that is located in the same chassis, no configuration is necessary on the Catalyst Supervisor. However, you must specify each MLS-capable router when using external routers. To do this, use the set mls include {route_processor_ip | route_processor_name} command. For example, to include an external router using the IP address 10.1.1.1, you should enter set mls include 10.1.1.1. Only include the IP address associated with the first MLS interface on the router.
Tip
The address to include is displayed on the mls ip address field of the show mls rp router command. Be sure to enter show mls rp on the router, not the Catalyst Supervisor.
The switch supports a set mls [enable | disable]. However, because MLS is enabled by default (given that you have the proper hardware and software), this command is not necessary.
Using MLS
Because the routing switch form of Layer 3 switching is a fairly new technique to most network administrators, this section takes a look at some of the more important MLS commands.
The show mls Command
One of the most important commands is the show mls Catalyst command. Several options are available for this command. In its basic form, show mls displays output similar to that included in Example 11-11.
Example 11-11 Output of show mls on the Catalyst Supervisor
Cat-A> (enable) show mls Multilayer switching enabled Multilayer switching aging time = 256 seconds Multilayer switching fast aging time = 0 seconds, packet threshold = 0 Current flow mask is Destination flow Configured flow mask is Destination flow Total packets switched = 0 Active shortcuts = 0 Netflow Data Export disabled Netflow Data Export port/host is not configured. Total packets exported = 0 MLS-RP IP MLS-RP ID XTAG MLS-RP MAC-Vlans ---------------- ------------ ---- --------------------------------- 10.1.1.1 00000c111111 2 00-00-0c-11-11-11 1 00-00-0c-22-22-22 2 The first three lines after the initial prompt tell you if MLS is enabled on this Catalyst and the configured aging timers. The next two lines indicate the flow mask currently in use and if a minimum flow mask has been configured. The Total packets switched and Active shortcuts lines can be very useful for keeping track of the amount of shortcut switching being performed and the size of your shortcut cache (as mentioned earlier, it is best to keep this value below 32,000 entries or current versions of the NFFC). The next three lines report the status of NetFlow Data Export, a feature that is discussed later. The bottom section lists all of the known routers, their XTAG values, and a list of the MAC addresses and VLANs.
The show mls entry Command
The show mls entry command is very useful when you want to examine the shortcut cache entries. Example 11-12 shows some sample output for the show mls entry command.
Example 11-12 Output of show mls entry
Cat-A> (enable) sh mls entry Last Used Last Used Destination IP Source IP Prot DstPrt SrcPrt Destination Mac Vlan Port --------------- --------------- ---- ------ ------ ----------------- ---- ----- MLS-RP 10.1.1.1: 10.1.1.9 10.1.2.99 TCP 1293 5001 00-60-08-b6-49-84 1 3/9 10.1.1.7 10.1.2.99 ICMP - - 00-60-08-b6-4a-49 1 3/13 10.1.2.99 10.1.1.7 ICMP - - 00-10-7b-3a-7b-97 2 3/18 Because the flow mask is set to destination, the cache only creates a single entry per destination address. Each cache entry is shown on a separate line. The Last Used Source IP, Protocol, Destination Port, and Source Port Fields show the characteristics of the packet that most recently used this shortcut entry. Because a single entry exists for all source nodes, protocols, and applications targeted to the destination address listed in the first column, it cannot list every type of packet individually (use a full flow mask for that level of detail).
If your cache is large, you probably want to use one of the options to filter the output. The full syntax for the show mls entry command is:
show mls entry {[destination ip_addr_spec] [source ip_addr_spec] | [flow protocol [ccc] src_port dst_port]} [rp ip_addr]
For example, show mls entry rp 10.1.1.1 lists all of the cache entries created from router 10.1.1.1. show mls entry destination 10.1.2.20lists the entries created for packets containing a destination IP address of 10.1.2.20.
The show mls statistics entry Command
Packet and byte counts for each entry can be listed with show mls statistics entry. The output is similar to that of show mls entry except it includes two new fields at the end of each line as shown in Example 11-13.
Example 11-13 Output of show mls statistics entry
Cat-A> (enable) show mls statistics entry Destination IP Source IP Prot DstPrt SrcPrt Stat-Pkts Stat-Bytes --------------- --------------- ---- ------ ------ ---------- --------------- MLS-RP 10.1.1.1: 10.1.2.99 10.1.1.7 ICMP - - 6 456 10.1.1.9 10.1.2.99 ICMP - - 8 824 10.1.1.7 10.1.2.99 TCP 1037 Telnet 11 802 The show mls statistics protocol Command
If you are using a full flow mask, the show mls statistics protocol command can provide extremely useful application layer information such as that displayed in Example 11-14.
Example 11-14 Output of show mls statistics protocol
Cat-A> (enable) show mls statistics protocol Protocol TotalFlows TotalPackets TotalBytes ---------- ---------- ------------ -------------- Telnet 6 24 1641 FTP 2 6 390 WWW 8 30 5219 SMTP 2 6 390 X 0 0 0 DNS 2 6 390 Others 12 11543 9093664 Total 32 11615 9101694 This information is very similar to the output of the show ip cache flow router command. By listing traffic volumes by both packet and byte counts, it can be very useful for profiling your network.
The debug mls rp Command
On the router, debug mls rp commands can be used to troubleshoot various issues. Example 11-15 displays the available options.
Example 11-15 Available Options for the debug mls rp Command
Router# debug mls rp ? all mls all error mls errors events mls events ip mls ip events locator mls locator packets mls packets verbose packets mls verbose packets As always, you should be very careful when using debug on production networks.
MLS Design Considerations
As discussed earlier, MLS can be characterized as a mechanism that caches "to the router and back" flows. Although this is a simple concept, it can be tricky to achieve in certain topologies and networks.
Tip
Note in advance that almost all of the issues discussed in this section can be avoided by simply making sure that every NFFC is paired with its own "internal router" such as the RSM (or RSFC). Because this automatically creates a "to the router and back" set of flows (across the blackplane of the Catalyst), it can dramatically simply your overall design considerations.
WAN Links
For example, MLS currently cannot be used on WAN links. Consider the network illustrated in Figure 11-13.
Figure 11-13. WAN Flows Defeat MLS
As with the earlier examples, Host-A is sending packets to Host-B. Recognizing that Host-B is on a different subnet, Host-A forwards all of the traffic to its default gateway, Router-A. The NFFC in Cat-A recognizes the first packet as a candidate packet and creates a partial shortcut entry. However, an enable packet is never received by Cat-A because the traffic is forwarded directly out the router's serial interface. The to the router and back flow necessary for MLS is not present. Because the shortcut entry is never completed, it ages out using the five-second quick aging scheme.
Using Multiple Router Ports
Although it is fairly obvious why MLS is not suitable for situations such as Figure 11-13, other situations can be far more subtle. The rule to remember is that the same NFFC must see the flow traveling to and from the router. For this to happen, the Catalyst performing MLS needs to participate in both VLANs/subnets. For example, the Catalysts shown in Figure 11-14 only contain a single VLAN.
Figure 11-14. Each Catalyst Contains Only a Single VLAN
The results in Figure 11-14 are very similar to those in Figure 11-13. Cat-A sees the candidate packet, but only Cat-B sees the enable packet. Shortcut switching is not possible.
Tip
MLS requires that the same NFFC or MSFC/PFC must see the flow traveling to and from the router. This can require careful planning and design work in certain situations.
However, simply placing both VLANs on both switches does not necessarily solve the problem. In Figure 11-15, both Cat-A and Cat-B contain the Red and Blue VLANs. An ISL trunk has even been provided to create a contiguous set of Layer 2 bridge domains.
Figure 11-15. Although Both Switches Contain Both VLANs, MLS Is Not Possible
However, because non-trunk links are used to connect the router, the router only sends and receives traffic for the Red VLAN to/from Cat-A, whereas all Blue traffic flows to/from Cat-B. The effect of this is the same as in the previous two examples: Cat-A only sees the candidate packet because the enable packet is sent to Cat-B.
A Solution for Using Multiple Router Ports
Although several alternatives are available to fix this configuration, the simplest option involves connecting the router to a single Catalyst. At this point, the one-link-per-VLAN or the trunk-connected router designs are appropriate. For example, Figure 11-16 simply swings the interface Ethernet1 router link from Cat-B to Cat-A to employ the one-link-per-VLAN design.
Figure 11-16. Connecting the Router to a Single Catalyst Permits MLS
As shown in Figure 11-16, this forces all inter-VLAN traffic to flow through Cat-A and, therefore, makes shortcut switching possible.
The Spanning-Tree Protocol and MLS
But what if additional switches are added so that additional Layer 2 loops are introduced? For example, consider the network shown in Figure 11-17 (assume all three Catalysts are MLS-capable and the router is connected via an ISL link).
Figure 11-17. A More Complex Network Using MLS
Because this is a redundant (that is, looped) Layer 2 topology, the Spanning-Tree Protocol becomes involved. Figure 11-17 assumes that Cat-A is functioning as the Root Bridge for all VLANs. This places one of the ports on Segment 3 in the Spanning Tree Blocking state. As a result, traffic flowing in the Red VLAN from Host-A to the router uses Segment 1. Both Cat-B and Cat-A recognize this as a candidate packet and create a partial shortcut entry. However, because traffic flowing from the router to Host-B uses Segment 2, only Cat-A sees the enable packet and creates a full shortcut entry. Cat-B's partial shortcut entry ages out in five seconds.
Consider what happens if Cat-B becomes the Spanning Tree Root Bridge. Figure 11-18 provides a diagram for this situation.
Figure 11-18. Cat-B Is the Spanning Tree Root Bridge
This causes Spanning Tree to reconverge to a logical topology where one of the ports on Segment 2 is Blocking. This allows the traffic from Host-A to the router to follow the same path as in Figure 11-17. Both Cat-A and Cat-B recognize the first packet as a candidate packet and create a partial shortcut entry. However, the traffic flowing from the router to Host-B cannot use Segment 2 because it is blocked. Instead, the traffic flows back through Cat-B and uses Segment 1 and Segment 3. Notice that this causes both Cat-A and Cat-B to see the Enable Packet and complete the shortcut entry.
When the second packet is sent from Host-A to Host-B, Cat-B uses its shortcut entry to Layer 3 switch the packet directly onto Segment 3, bypassing the router. Because Cat-A does not see any traffic for the shortcut entry it created, the entry ages out in 256 seconds by default. Although this allows MLS to function (in fact, it creates a more efficient flow in this case), it can be disconcerting to see the shortcut switching operation move from Cat-A to Cat-B only because of Spanning Tree. Obviously, the interaction between MLS and Spanning Tree can get very complex in large and very flat campus networks (yet one more reason to avoid the flat earth approach to campus design; see Chapters 14 and 15 for more information).
Using MLS on a Subset of Your Switches
Figures 11-17 and 11-18 assumed that all three Catalysts supported MLS. Subtleties can also arise if this is not the case. For example, what if Cat-A did not support MLS? In Figure 11-17, MLS would not be possible. Because Cat-A was the only switch to see both sides of the flow, it is the only device capable of performing MLS in this topology. However, MLS is possible in Figure 11-18, even if Cat-A is not MLS-capable. Because of the Spanning Tree reconvergence performed in Figure 11-18, Cat-B now sees both sides of the flow and can perform shortcut switching in the same manner as already explained.
Using Stacks of MLS Devices
This section considers the case where stacks of MLS-capable devices connect multiple end stations and Catalysts as shown in Figure 11-19.
Figure 11-19. Host-A Communicating with Host-B via MLS
First, look at the case of Host-A sending traffic to Host-B. The traffic from Host-A to the router travels up the ISL links connecting the Catalysts and the router to each other. As the first packet hits the NFFC in each Catalyst, it is recognized as a candidate packet and three partial shortcut entries are created (one per Catalyst). As the packet travels back down from the router to reach Host-B, all three NFFC cards see the enable packet and complete the shortcut entries. However, as additional packets travel from Host-A to Host-B, Cat-A shortcut switches them directly to Host-B. The shortcut entries in Cat-B and Cat-C simply age out in 256 seconds (by default).
Now consider the flow from Host-A to Host-C. Again, all three NFFCs see the initial packet as a candidate packet. However, the return packet only passes through Cat-C and Cat-B. The partial shortcut entry in Cat-A ages out in five seconds. As Host-A sends additional packets, Cat-A uses normal Layer 2 switching to send the packets towards the MAC address of the router. When Cat-B receives the packets, it recognizes that it has a completed shortcut for this flow and shortcut switches the packets directly to Host-C. Cat-C's shortcut entry is not used and therefore ages out in 256 seconds. Figure 11-20 illustrates this sequence.
Figure 11-20. Host-A Communicating with Host-C via MLS
Using Multiple Routers with a Single MLS-Capable Catalyst
Finally, consider the case of multiple MLS-capable routers and a single MLS switch shown in Figure 11-21.
Figure 11-21. Two MLS Routers and One MLS Switch
Here, Host-A is still located in the Red VLAN and Host-B is still located in the Blue VLAN. However, a new VLAN has been created between the two routers (call it the Purple VLAN). Host-A still sends traffic destined to Host-B to its default gateway using the Red VLAN. As the first packet passes through the Catalyst, the NFFC recognizes it as a candidate packet and creates a partial shortcut entry (labeled Step 1 in Figure 11-21). Router-A then forwards the traffic over the Purple VLAN to Router-B. As the packet passes back through the Catalyst, the NFFC recognizes the packet as an enable packet and completes the shortcut entry (Step 2 in Figure 11-21). However, it also recognizes the destination MAC address as that of Router-B and therefore sees this packet as another candidate packet (Step 3 in Figure 11-21). Router-B then routes the packet normally and forwards it to Host-B over the Blue VLAN. As the packet passes back through the Catalyst for the third time, it is identified as an enable packet for the partial entry created in Step 3. A second shortcut entry is created (Step 4 Figure 11-21).
When additional traffic flows from Host-A to Host-B (Step 5 in Figure 11-21), two sets of shortcut lookups and rewrite operations are performed. As a result, the additional packets are not sent to either router. Neat!
Other MLS Capabilities
One of the nicest things about using the NFFC for MLS is that it enables many other applications that extend beyond accelerated Layer 3 routing. Because the NFFC is at its most basic level a pattern-matching engine, these pattern-matching capabilities can be used to provide many interesting and powerful features as detailed in the following sections.
Protocol Filtering
Protocol Filtering is the capability of the NFFC to limit broadcast and multicast traffic on a per-port and per-protocol basis. As discussed in Chapter 5, "VLANs," it allows a group of nodes to be placed in a single VLAN and only receive traffic associated with the protocols they are actually running. Four groupings of protocols exist: IP, IPX, a combined group of AppleTalk and DECnet (some platforms also include VINES here), and a final group that contains all other protocols. By pattern matching on the protocol type information contained in the Layer 2 header, the NFFC can, for example, filter IPX SAPs on ports that are only using IP.
Protocol Filtering is disabled by default. To enable this feature, use the set protocolfilter enable command. To configure Protocol Filtering, use the set port protocol command:
set port protocol mod_num/port_num {ip|ipx|group} {on|off|auto}
The group parameter corresponds to AppleTalk and DECnet (and, in some cases, VINES). The on state forces that port to send broadcasts of the specified type. The off state forces that port to not send broadcasts of the specified type. The auto state only send broadcasts for the specified protocol if that protocol is detected coming in that port. This creates a dynamic configuration where the Catalyst is detecting the protocols being run on each port and only sending the appropriate broadcasts in response. IP defaults to the on state, and the other protocol categories (IPX and "group") default to auto.
The show protocolfilter command can be used to determine if Protocol Filtering is running on a device. The show port protocol command can be used to view the configuration on a per-port basis (including the number of nodes detected on a per port and per protocol basis). For ports and protocols in the auto state, auto-on and auto-off are used to indicate the dynamically selected setting currently in use. Trunk ports are excluded from Protocol Filtering.
Multicast Switching and IGMP Snooping
Traditional transparent bridges (and, therefore, most Layer 2 switches) do not have a mechanism to learn multicast MAC address. As a result, these Layer 2 bridging and switching devices treat multicast frames as if they were broadcast frames. Although this does allow multicast applications to function, it has the highly undesirable side effect of wasting lots of bandwidth (it can also waste CPU cycles on end stations).
One solution to this problem is the use of static CAM entries. However, given the growing popularity of multicast usage, this can rapidly become a huge management problem. For example, every time a user wants to join or leave a multicast group, it requires manual intervention by the network administrators. In a large network, this can easily amount to hundreds of entries and adjustments per day.
Clearly some sort of dynamic process is required. Three options are available for dynamically building multicast forwarding tables: CGMP, GMRP, and IGMP Snooping. This section briefly discusses these three options, especially as they pertain to the NFFC. For a more thorough discussion, please see Chapter 13.
Of these techniques, Cisco developed the Cisco Group Management Protocol (CGMP) first. This allows routers running the Internet Group Management Protocol (IGMP) to update the Catalyst Layer 2 CAM table. IGMP is a protocol that allows end stations to request that routers send them a copy of certain multicast streams. However, because it is a Layer 3 protocol, it is difficult for a Layer 2 switch to speak this protocol. Therefore, Cisco developed CGMP. Think of it as a mechanism that allows a Layer 3 router to tell a Layer 2 Catalyst about multicast group membership. As a result, the Layer 2 Catalyst forwards IP multicast traffic only to end-station ports that are actually interested.
Configuring CGMP on a Catalyst is simple. It runs by default on most Catalysts, requiring no configuration whatsoever. Other Catalysts, such as the 5000, require the set cgmp enable command. The show multicast group cgmp command can be used to display the multicast MAC address to port mappings created via the CGMP protocol. To configure CGMP on the router, the ip cgmp command must be configured on the interfaces where CGMP support is desired. In addition, some sort of multicast routing protocol must be configured (PIM dense-mode is the simplest option).
In the future, the GARP Multicast Registration Protocol (GMRP) might become a commonly used approach. GMRP uses the Generic Attributed Registration Protocol (GARP) specified in 802.1p to provide registration services for multicast MAC addresses. However, because work is still ongoing in the development of GMRP, this is not a suitable option today.
The third option, IGMP Snooping, is a standards-based alternative to the Cisco Group Management Protocol (CGMP). This relies on the pattern-matching capabilities of the NFFC to listen for IGMP packets as they flow between the router and the end stations. By inspecting these packets, the Catalyst can learn which ports have end stations interested in which multicast groups.
Some vendors have implemented IGMP Snooping using general-purpose CPUs. However, without some sort of hardware-based support, this approach suffers from extreme scaling problems. This situation arises because IGMP messages are intermixed with data in literally every multicast flow in the network. In short, vendors cannot simply point a single IGMP multicast MAC address at the CPU. Instead, the switch must sort through every packet of every multicast stream looking for and processing IGMP packets. Do not try this on a general-purpose CPU!
Note
This also suggests that IGMP is not a replacement for CGMP. IGMP is suitable for high-end devices that contain ASIC-based pattern-matching capabilities. However, low-end devices without this support still require the services of CGMP. In fact, many multicast networks require both.
The good news is that IGMP Snooping is extremely easy to configure. Because IGMP Snooping is a passive listening process much like routed running in quiet mode on a UNIX box, no configuration is required on the router (although it still needs to be running a multicast routing protocol). On the Catalyst, simply enter the set igmp enable command. Use the show multicast group igmp command to display the list of multicast MAC address to port mappings created via the IGMP Snooping process.
Quality of Service
Although the initial version of the NFFC (NFFC I) did not support Quality of Service (QoS) and Class of Service (COS), more recent versions (NFFC II and MSFC/PFC) have included this feature. This capability is targeted at being able to reclassify traffic in the wiring closet at the edge of the network. This can allow mission critical traffic to be flagged as such using Layer 3 IP Type of Service (ToS) bits or Layer 2 capabilities such as 802.1p and ISL (the ISL header contains 3 bits for COS). Devices that support sophisticated queuing and scheduling algorithms such as the Catalyst 8500 can then act on these QoS/COS fields to provide differentiated service levels. Because these capabilities are still evolving at the time this book goes to press, they are not discussed here.
NetFlow Data Export
Just as MLS show commands such as show mls entry and show mls statistics protocol provide incredibly detailed information on protocol flows in the network, this information can be captured on an ongoing basis with NetFlow Data Export (NDE). NDE ships information on each flow to a NetFlow collection device (such as Cisco's NetFlow FlowCollector). These collection stations then massage the data to eliminate duplicate information (a single flow could have traversed and therefore been reported by several devices) and output information such as reports and billing information. First enable NDE with set mls nde enable. Then specify the IP address and port number of the collector station with set mls nde {collector_address | collector_name} port_number.
Tip
The set mls nde flow command can be used to filter the amount of information collected by NDE. For example, set mls nde flow destination 10.1.1.1/32 source 10.1.1.2/32 collects information flowing only from 10.1.1.2 to 10.1.1.1.
When to Use MLS
MLS's hardware-assisted approach to routing can be very useful when the RSM and router-on-a-stick techniques do not have enough capacity. In fact, one of the most appealing benefits to MLS is that it can be easily added to an existing network to turbocharge the routing performance. Therefore, the most common argument for using MLS is throughput. The additional capabilities of the NFFC to handle tasks such as Protocol Filtering, IGMP Snooping, NetFlow data collection, and QoS can make MLS an even more attractive option than simply using it for go fast routing. However, to fully discuss the pros and cons of MLS, 8500-style routing must be considered.
EAN: N/A
Pages: 223