Tips and Tricks: Mastering STP

Tips and Tricks: Mastering STP

This section summarizes some of the advice given in earlier sections while also introducing some other STP best practices.

Manually position Root Bridges. Leaving Root Bridge placement to chance can dramatically affect the stability and scalability of your network. For more information, see the sections "Using Spanning Tree in Real-World Networks" and "Deterministic Root Bridge Placement" in Chapter 6.

Always have at least one backup Root Bridge. If you design your network around a single Root Bridge, all of your carefully laid plans can unravel when the Root Bridge fails. All Layer 2 networks should have at least one backup Root Bridge. Large networks might require more than one.

Try to locate Root Bridges on devices near heavily used traffic destinations. For flat networks lacking sufficient Layer 3 hierarchy, this usually means placing the Root Bridges for all VLANs on a pair of redundant switches at the server farm entrance. For more hierarchical networks, collocate the Root Bridges with the routers acting as the default gateways for the end-user segments (see Chapters 11, 15, and 17 for more information).

Diagram your primary and backup topologies. Most network managers recognize the value in having network diagrams. However, most of these diagrams only show the Layer 3 topology. Furthermore, tools such as HP OpenView tend to be very Layer 3 centric (although this is starting to change). Unfortunately, these Layer 3 diagrams tell you nothing about your Layer 2 topology it all appears as one big subnet. CiscoWorks for Switched Internetworks (CWSI) in CiscoWorks 2000 has a simplistic but effective STP mapping tool.

Network managers running a large switched infrastructure should consider placing the same care and effort into Layer 2 diagrams as they do with Layer 3 diagrams. When doing this, be sure to capture the primary and back up active Spanning Tree topologies. The diagram should indicate which ports are Forwarding and which ports are Blocking for each VLAN. Knowing this ahead of time can be a huge lifesaver when the network is down. It can be confusing enough just to figure out your STP topology on a calm day trying to figure it out when the network is down is no fun at all!

Tip

Many CWSI/CiscoWorks 2000 users are not aware that it contains a Spanning Tree mapping tool. To use it, first pull up VLAN Director. Then select a VTP domain. Then pick a VLAN in that domain. This highlights the nodes and links that participate in the VLAN. It also brings up the VLAN section (it has a yellow light bulb next to it). Click the Spanning Tree checkbox and the Blocking ports, which are marked with a sometimes-hard-to-see X.

Update your design after adding to the network. After carefully implementing load balancing and Root Bridge failover strategies, be sure to evaluate the impact of network additions and modifications. By adding devices and paths, it is easy for innocent-looking changes to completely invalidate your original design. Also be sure to update your documentation and diagrams. This is especially true if you are using a flat-earth design (see the section "Campus-Wide VLANs Model" in Chapter 14).

Avoid timer tuning in flat designs. Unless you have a narrow Spanning Tree diameter and a very controlled topology, timer tuning can do more harm than good. It is usually safer and more scalable to employ techniques such as UplinkFast and BackboneFast.

Max Age tuning is less risky than Forward Delay tuning. Although overly-aggressive Max Age tuning can lead to excessive Root Bridge, Root Port, and Designated Port elections, it is less dangerous than overly-aggressive Forward Delay tuning. Because Forward Delay controls the time a device waits before placing ports in the Forwarding state, a very small value can allow devices to create bridging loops before accurate topology information has had time to propagate. See the sections "Tuning Max Age" and "Tuning Forward Delay" earlier in this chapter for more information.

If you do resort to timer tuning, consider using the set spantree root macro. This macro sets Spanning Tree parameters based on the recommended formulas at the end of the 802.1D spec. For more information, see the section "Using A Macro: set spantree root" in Chapter 6.

Use timer tuning in networks using the multilayer design model. Because this approach constrains the Layer 2 topology into lots of Layer 2 triangles, consider using STP timer tuning in networks using the multilayer design model. In general, it is recommended to use the set spantree root command and specify a diameter of 2 3 hops and a Hello Time of two seconds. See the section "Timer Tuning" in Chapter 15 for more detailed information.

Utilize Root Bridge placement load balancing in networks employing MLS and the multilayer design model. This chapter discussed the importance of using Layer 3 switching to limit the size of your Spanning Tree domains. Chapters 11, 14, 15, and 17 look at the use of various approaches to Layer 3 switching and make some specific recommendations on how to use this technology for maximum benefit. Chapter 14 details one of the most successful campus design architectures, the so-called multilayer model. When implemented in conjunction with Cisco's MLS/NFFC, the multilayer model seeks to reduce Spanning Tree domains to a series of many small Layer 2 triangles. Because these triangles have very predictable and deterministic traffic flows, they are very well suited to using the Root Bridge Placement form of STP load balancing. In general, the Root Bridges should be located near or collocated with the default gateway router for that VLAN.

Port/VLAN cost is the most flexible STP load balancing technique. Other than Root Bridge placement, which can be useful in networks with well-defined traffic patterns for each VLAN, port/VLAN cost load balancing is the preferable option. In general, it should be used in all situations other than the case mentioned in the previous tip. For details, see "Load Balancing with Port/VLAN Cost" earlier in this chapter.

Spanning Tree load balancing requires that multiple VLANs be assigned to IDF wiring closet switches. Although assigning a single VLAN to IDF switches can make administration easier, it prevents STP load balancing from being possible. In some cases, non-STP load balancing techniques such as MHSRP and EtherChannel might still be possible with a single IDF VLAN.

Use a separate management VLAN. Just like end-stations, Catalysts must process all broadcast packets in the VLAN where they are assigned. In the case of a Layer 2 Catalyst, this is the VLAN where SC0 is placed. If this VLAN contains a large amount of broadcast traffic, it can then overload the CPU and cause it to drop frames. If end-user traffic is dropped, no problem. However, if STP (or other management) frames are dropped, the network can quickly destabilize. Isolating the SC0 logical interfaces in their own VLAN protects them from end-user broadcasts and allows the CPU to focus only on important management traffic. In many cases, the factory default VLAN (VLAN 1 for Ethernet) works well as the management VLAN. Chapter 15 discusses management VLAN design issues and techniques.

Minimize Layer 2 loops in the management VLAN. Many networks contain lots of redundancy in the management VLAN. The thought is that it prevents failovers from isolating switch management capabilities. Unfortunately, this can also destabilize the network. In networks with lots of management VLAN loops, all it takes is a single switch to become overloaded or run into an STP bug. If this switch then opens up a bridging loop in the management VLAN, suddenly neighboring bridges see a flood of broadcast and multicast traffic. As this traffic inevitably overloads the neighboring switches, they can create additional bridging loops. This phenomenon can pass like a wave across the entire network with catastrophic results. Although it might only directly effect VLAN 1, by disabling the CPUs in all bridges and switches, it effectively shuts down the entire Layer 2 network. To avoid these problems, it is advisable to use Layer 3 routing, not Layer 2 bridging, to provide redundancy in the management VLAN. This point is discussed further in Chapter 15.

Use Layer 3 switching to reduce the size of Spanning Tree domains. Now that you are armed with a truck-load of STP knowledge, creating scalable and flexible STP designs should be a breeze! However, this knowledge should also lead you to the conclusion that excessively large Spanning Tree domains are a bad idea. Small STP domains provide the best mix of failover performance and reliability. See Chapters 11, 14, 15, and 17 for more information.

Try to design your network such that Spanning Tree domains consist of MDF-IDF-MDF triangles. This maximizes STP's value as a Layer 2 failover feature minimizes any scalability concerns.

Use PortFast on end-station ports to reduce Topology Change Notifications. Not only can PortFast eliminate problems associated with devices that boot and access the network quickly, it reduces the number of Topology Change Notifications in the network. See the "PortFast" section in the chapter for more information.

Use UplinkFast to improve IDF wiring closet failover time. UplinkFast is an extremely effective and clever STP optimization that reduces most wiring closet failover times to two or three seconds. See the "UplinkFast" section in this chapter for more information.

PVST+ load balancing requires all inter-switch ports on MST switches to be in the Forwarding state. PVST+ allows traditional PVST Catalysts to interoperate with 802.1Q switches that only use a single Spanning Tree (MST). Best of all, it does this without any additional configuration! However, it might require careful planning to maintain effective STP load balancing. See the "PVST+" section of this chapter for more information.

Always specify the VLAN parameter with Spanning Tree commands to avoid accidental changes to VLAN 1. Many of the Spanning Tree set and show commands allow you to omit the VLAN parameter. When doing so, you are implying VLAN 1. To avoid confusion and unintentional modifications to VLAN 1, it is best to get in the habit of always specifying this parameter.

The original implementations of Fast EtherChannel in 2.2 and 2.3 NMP images did not support STP over EtherChannel. Spanning Tree still viewed the link as two or four separate ports and would block all but one (obviously defeating the purpose of EtherChannel). The limitation was solved in 3.1 and later versions of code. Don't be misled by older or incorrect documentation that does not reflect this enhancement using STP over EtherChannel is generally a good idea.

Utilize the "desirable" EtherChannel mode for maximum Spanning Tree stability. When using the on mode, it is possible for the entire channel to be declared down when only the single link carrying STP failed. See Chapter 8 for more information on EtherChannel technology.

Be certain that you do not overload your CPU with Spanning Tree calculations. Keep the total number of logical ports below the values specified in Table 7-4.

Table 7-4. Maximum Logical Ports

Cat 5000 Supervisor

Max Logical Ports

Supervisor I

400

Supervisor II

1500

Supervisor III

3500

Use the following formula to calculate the number of logical ports on your devices:

Logical Port = number VLANs on non-ATM trunks + (2 x number VLANs on ATM trunks) + number non-trunk ports

In other words, you want to add up the total number of VLANs on every port in your box. ATM VLANs (these are called ELANs; see Chapter 9, "Trunking with LAN Emulation") are more heavily weighed by counting them twice.

For example, consider a Catalyst 5000 MDF switch with 100 Ethernet trunk ports, each of which carry 25 VLANs. Also assume that the MDF switch is 32 Ethernet-attached servers using non-trunk links. In this case, the total number of logical ports would be:

2,532 = (100 trunks x 25 VLANs) + 32 non-trunk ports

Although this is by no means the largest MDF switch possible with Catalyst equipment, notice that it requires a Supervisor III. If the trunks were ATM trunks, the total number of logical ports swell to 5,032 more than even a Catalyst 5000 Supervisor III can handle.

Finally, note that this calculation assumes a Hello Time of two seconds. If you have decreased your Hello Time to one second to speed convergence, double the value you calculate in the formula. For instance, the number of Ethernet logical ports in the previous example would be 5,032, and the number of ATM logical ports would swell to 10,064!