Spanning Tree Load Balancing
The previous section discussed the concepts of PVST and how it allows Spanning Tree isolation and control. One of the most important benefits of this isolation and control is load balancing, the capability to utilize multiple active paths.
There is both good and bad news associated with load balancing. The good news is that it can be used to effectively double your network's throughput. The bad news is that it can triple the complexity to your network! Although I might be exaggerating a bit, the added complexity can be especially burdensome in a poorly designed network (see Chapters 14 and 15 for more information). This section takes a detailed look at the techniques that are available for implementing this advanced feature. By examining the theory behind each feature, you will be well equipped to maximize your available bandwidth with minimal added complexity.
It is important to realize that the term load balancing is a bit of a euphemism. In reality, Spanning Tree load balancing almost never achieves an even distribution of traffic across the available links. However, by allowing the use of more than one path, it can have a significant impact on your network's overall performance. Although some people prefer to use terms such as load distribution or load sharing, this text uses the more common term of load balancing.
There are four primary techniques for implementing Spanning Tree load balancing on Catalyst gear. Table 7-1 lists all four along with the primary command used to implement each.
Table 7-1. Techniques for Spanning Tree Load Balancing
Technique
Command
Root Bridge Placement
set spantree priority
Port Priority
set spantree portvlanpri
Bridge Priority
set spantree priority
Port Cost
set spantree portvlancost
Each of these is discussed in the separate sections that follow. Note that the discussion only addresses load balancing techniques based on the Spanning-Tree Protocol. For a discussion of non-STP load balancing techniques, see Chapter 11 and Chapter 15.
General Principles of Spanning Tree Load Balancing
Spanning Tree load balancing requires that two characteristics be built into the network:
Multiple paths that form loops
Multiple VLANs
If the network doesn't contain loops, there cannot be multiple paths over which the load can be distributed. By way of illustration, if two switches only have one Fast Ethernet link between them, it is fairly difficult to do any load balancing. This concept is obviously tightly coupled with the desire to have redundancy in campus networks. For example, most networks employ at least two links to every IDF wiring closet to meet reliability and availability requirements even if the bandwidth requirements are low. However, after the redundant paths are available, the question becomes, "How do you make use of them?" In fact, load balancing is all about having the best of both worlds redundancy and speed.
However, if loops exist in the network, isn't Spanning Tree designed to remove them from the active topology? In other words, how can you use multiple paths if Spanning Tree prunes the network back to a single set of paths that reach from every location to every other location? Well, with only one Spanning Tree domain, you can't. On the other hand, VLANs make it possible to create multiple Spanning Tree domains over a single physical infrastructure. By intentionally designing different active topologies into each VLAN, you can utilize multiple redundant paths. Within a single VLAN, the traffic is loop free and only utilizes a single path to reach each destination. But two VLANs can use both redundant links that you have installed to a wiring closet switch.
For example, consider Figure 7-9, a variation of the simplified campus network used in Figure 7-2.
Figure 7-9. A Simplified Campus Network
In this version of the simplified campus network, the campus backbone has been eliminated so that the entire network is contained within a single building. Instead of locating the server farm in a separate building, the servers have been moved to the MDF closets. Redundant links have been provided to the IDF switches (only one IDF is shown). Assume that the wiring closet switch only supports 20 users that rarely generate more than 100 Mbps of traffic. From a bandwidth perspective, a single riser link could easily handle the load. However, from a redundancy perspective, this creates a single point of failure that most organizations are not willing to accept. Given the assumption that corporate policy dictates multiple links to every wiring closet, wouldn't it be nice to have all the links available for carrying traffic? After all, after both links have been installed, Spanning Tree load balancing can potentially double the available bandwidth for free! Having three paths to every closet is indicative of two things: You work for a really paranoid organization and you can possibly triple your bandwidth for free!
Even when redundant links are available, don't overlook the second STP requirement for Spanning Tree load balancing multiple VLANs. Many organizations prefer to place a single VLAN on each switch to ease VLAN administration. However, this is in conflict with the goal of maximizing bandwidth through load balancing. Just remember one VLAN=one active path. This is not meant to suggest that it's wrong to place a single VLAN on a switch, just that it prevents you from implementing Spanning Tree load balancing.
Tip
If you only place a single VLAN on a switch, you cannot implement Spanning Tree load balancing. Either add VLANs or consider other load balancing techniques such as EtherChannel or MHSRP. See Chapter 11 and Chapter 15 for more information.
Root Bridge Placement Load Balancing
Fortunately, one of the most effective load balancing techniques is also the simplest. By carefully placing the Root Bridge for each VLAN in different locations, data can be forced to follow multiple paths. Each VLAN looks for the shortest path to its Root Bridge.
For example, Figure 7-10 adds load balancing to the network in Figure 7-9.
Figure 7-10. Load Balancing through Root Bridge Placement
This network contains two VLANs. Cat-A is the Root Bridge for VLAN 2, and Cat-B is the Root Bridge for VLAN 3. From Cat-C's perspective, the available bandwidth to the servers has been doubled. First, examine VLAN 2. Cat-C has two possible paths to the Root Bridge: Cat-C:Port-1/1 can reach the Root Bridge with a cost of 19, whereas Cat-C:Port-1/2 can get there at a cost of 38. Obviously, Port 1/1 is chosen as Cat-C's Root Port for VLAN 2. VLAN 3 also has two paths to the Root Bridge, but this time the costs are reversed: 38 through Port 1/1 and 19 through Port 1/2. Therefore, VLAN 3's traffic uses Cat-C:Port-1/2. Both links are active and carrying traffic. However, if either link fails, Spanning Tree places all bandwidth on the remaining link to maintain connectivity throughout the network.
Root Bridge Placement in Hierarchical Networks
The advantage of Root Bridge placement load balancing is that it can provide a simple yet effective increase in the network's aggregate bandwidth. This is especially true with networks that utilize Layer 3 switching in designs such as the multilayer model discussed in Chapters 14 and 15. In this case, the Root Bridges should simply be co-located (or located near) the routers serving as the default gateways. For example, Figure 7-11 uses this approach to load balancing traffic for VLANs 2 and 3.
Figure 7-11. Co-locating Root Bridges with Default Gateways
Not only has the load been distributed between the two routers (Router A is handling VLAN 2, whereas Router B is handling VLAN 3), the Layer 2 load has also been spread across both IDF uplinks. By placing the Root Bridge for VLAN 2 on Cat-A, the Spanning-Tree Protocol automatically creates the best Layer 2 path to the default gateway, the first-hop destination of most traffic in modern campus networks. The same has been done with Cat-B and VLAN 3.
More information on this technique (including how to use HSRP for redundancy) is discussed in Chapter 11.
Root Bridge Placement in Flat Networks
Although Root Bridge placement is appropriate for most well-designed campus networks such as that shown in Figure 7-11, the disadvantage is that many topologies do not support this approach. This is especially true of non-hierarchical, flat-earth networks. In some of these networks, traffic patterns are not clearly enough defined (or understood) for this technique to be effective. In other cases, the traffic between the VLANs is too similar.
As Chapter 15 discusses, the general rule of thumb in non-hierarchical networks is that you should try to place your Root Bridges in the path of your high-volume traffic flows. In the case of flat-earth networks, this usually translates into locating the Root Bridges as close to your servers as possible. Because end-station-to-server farm traffic accounts for over 90 percent of the load on most modern networks, this approach causes Spanning Tree to look for the shortest paths where your load is heaviest. If you place the Root Bridge at the other end of the network from your server farm, the high-volume traffic might be forced to take a very inefficient path.
Therefore, the Root Bridge placement load balancing technique is most effective in flat-earth networks where there is a clearly defined server location for every VLAN. Simply place the Root Bridge on the server farm switches and Spanning Tree naturally looks for the most efficient path. Figure 7-12 illustrates such a network.
Figure 7-12. Root Bridge Placement Load Balancing Requires Well-Defined Traffic Patterns
Part A of Figure 7-12 illustrates the physical topology and the location of servers. The Sales department has its servers attached to Cat-A, whereas the Human Resources department has connected their servers to Cat-F. Part B of Figure 7-12 shows the active topology for the Sales VLAN. By placing the Root Bridge at the servers, the Spanning Tree topology automatically mirrors the traffic flow. Part C of Figure 7-12 shows the active topology for the Human Resources VLAN. Again, the paths are optimal for traffic destined to the servers in that VLAN. Consider what happens if the Root Bridges for both VLANs are placed on Cat-F. This forces a large percentage of the Sales VLAN's traffic to take an inefficient path through Cat-F.
A potential problem with using this technique is that the traffic between the VLANs might be too similar. For example, what if a single server farm handles the entire network? You are left with two unappealing options. First, you could optimize the traffic flow by placing all of the Root Bridges at the server farm, but this eliminates all load balancing. Second, you could optimize for load balancing by distributing the Root Bridges, but this creates unnecessary bridge hops for traffic that is trying to reach the servers.
Implementing Root Bridge Placement
To implement Root Bridge placement load balancing, use the set spantree priority command discussed in the "Manual Root Bridge Placement: set spantree priority" section of Chapter 6. For instance, load balancing could be achieved in Figure 7-12 with the following commands:
Cat-A (enable) set spantree priority 100 2 Cat-F (enable) set spantree priority 100 3
The first command lowers the Bridge Priority on Cat-A to 100 for VLAN 2 (the Sales VLAN) so that it wins the Root Bridge election. In a similar fashion, the second command configures Cat-F to be the Root Bridge for VLAN 3 (the Human Resources VLAN).
Port/VLAN Priority Load Balancing
Another topology where Root Bridge placement load balancing does not work is with back-to-back switches or bridges. For example, consider Figure 7-13.
Figure 7-13. Back-to-Back Switches Cannot Use Root Bridge Placement Load Balancing
First, examine VLAN 2 as shown in Part A. Cat-A needs to pick a single Root Port to reach Cat-B, the Root Bridge for VLAN 2. As soon as Cat-A recognizes Cat-B as the Root Bridge, Cat-A begins evaluating Root Path Cost. Because both Cat-A:Port-1/1 and Cat-A:Port-1/2 have a cost of 19 to the Root Bridge, there is a tie. In an effort to break the tie, Cat-A considers the Sending BID that it is receiving over both links. However, both ports are connected to the same bridge, causing Cat-A to receive the same Sending BID (100.BB-BB-BB-BB-BB-BB) on both links. This results in another tie. Finally, Cat-A evaluates the Port ID values received in Configuration BPDUs on both ports. Cat-A:Port-1/1 is receiving a Port ID of 0x8001, and Cat-A:Port-1/2 is receiving a Port ID of 0x8002. Cat-A chooses the lower value for a Root Port, causing it to send all traffic out Port 1/1 while Blocking on Port 1/2.
With VLAN 3 in Part B of Figure 7-13, the roles are reversed but the outcome is the same all traffic is passing over the left link. In this case, it is Cat-B that must elect a Root Port to reach the Root Bridge, Cat-A. Cat-B calculates identical values for Root Path Cost and Sending BID as discussed in the previous paragraph. To resolve the resulting tie, Cat-B evaluates the received Port ID values, chooses Port 1/1 as the Root Port, and places Port 1/2 in the Blocking state.
Although the Root Port selection and traffic flow is happening in the opposing direction, both switches have selected Port 1/1 as the Forwarding link. The result the left link is carrying 100 percent of the traffic, and the right link is totally idle.
Note
Note that it is possible to implement load balancing in Figure 7-13 by crossing the cables such that Cat-B:Port-1/1 connects to Cat-A:Port-1/2 and Cat-B:Port-1/2 connects to Cat-A:Port-1/1. However, this approach is not very scalable and can be difficult to implement in large networks. Exercise 1 at the end of this chapter explores this load balancing technique.
Although the network in Figure 7-13 fails to implement load balancing, it does raise two interesting points. First, notice that it is the non-Root Bridge that must implement load balancing. Recall that all ports on the Root Bridge become Designated Ports and enter the Forwarding state. Therefore, it is the non-Root Bridge that must select a single Root Port and place the other port in a Blocking state. It is precisely this decision process that must be influenced to implement load balancing.
Second, it is the received values that are being used here. Cat-A is not evaluating its own BID and Port ID; it is looking at the values contained in the BPDUs being received from Cat-B.
Note
There is one case where the local Port ID is used. As shown in Figure 7-14, imagine two ports on Cat-B connecting to a hub that also connects to Cat-A, the Root Bridge. In this case, the received Port ID is the same on both ports of Cat-B. To resolve the tie, Cat-B needs to evaluate its own local Port ID values (the lower Port ID becomes the Root Port). This topology is obviously fairly rare in modern networks and not useful for load balancing purposes.
Figure 7-14. Using the Local Port ID Value As a Tie-Breaker
How can the load balancing be fixed in Figure 7-13? Given that Port ID is being used as decision criterion to determine which path to use, one strategy is to focus on influencing these Port ID values. On Catalysts using the XDI/CatOS interface (such as the Catalyst 4000s, 5000s, and 6000s), this can be done by applying the set spantree portvlanpri command. The full syntax for this command is
set spantree portvlanpri mod_num/port_num priority [vlans]
where mod_num is the slot number that a line card is using and port_num is the port on an individual line card.
As with the other set spantree and show spantree commands already discussed, the vlans parameter is optional. However, I suggest that you always code it so that you don't accidentally modify VLAN 1 (the default) one day.
The priority parameter can have the values ranging from 0 to 63, with 32 being the default. The priority parameter adjusts the Port ID field contained in every Configuration BPDU. Although the Port ID field is 16 bits in length, set spantree portvlanpri just modifies the high-order 6 bits of this field. In other words, Port ID consists of the two subfields shown in Figure 7-15.
Figure 7-15. Port ID Is a 16-Bit Field Consisting of Two Subfields
Port Number is a unique value statically assigned to every port: 1 for Port 1/1, 2 for Port 1/2, and so on (because number ranges are assigned to each slot, not all of the numbers are consecutive). Being 10 bits in length, the Port Number subfield can uniquely identify 210, or 1,024, different ports. The high-order 6 bits of the Port ID field hold the Port Priority subfield. As a 6-bit number, this subfield can mathematically hold 26=64 values (0 to 63, the same values that can be used with the set spantree portvlanpri command). Because the Port Priority subfield is contained in the high-order bits of Port ID, lowering this value by even one (to 31 from the default of 32) causes that port to be preferred.
The overly observant folks in the crowd might notice that Cisco routers use a different range of Port Priority values than do Catalyst switches. Whereas Catalysts accept Port Priority values between 0 and 63, the routers accept any value between 0 and 255. This difference comes from the fact that the routers are actually using the values specified in the 802.1D standard. Unfortunately, the 802.1D scheme only uses 8 bits for the Port Number field, limiting devices to 256 ports (28). Although this is more than adequate for traditional routers, it is a significant issue for high-density switches such as the Catalyst 5500. By shifting the subfield boundary two bits, the Catalysts can accommodate the 1,024 ports calculated in the previous paragraph (210). Best of all, this difference is totally harmless as long as every port has a unique Port ID, the Spanning-Tree Protocol is perfectly happy. In fact, as combined Layer 2/Layer 3 devices continue to grow in popularity and port densities continue to increase, this sort of modification to the 802.1D specification should become more common.
Note
Starting in 12.0 IOS, the routers started using a subfield boundary that allows the Port Number subfield to be between 9 and 10 bits in length. This was done to support high-density bridging routers such as the 8540. The new scheme still allows Port Priority to be specified as an 8-bit value (0 255) and then the value is divided by either two (9-bit Port Number) or four (10-bit Port Number) to scale the value to the appropriate size.
How does all this bit-twiddling cause traffic to flow across multiple paths? Figure 7-16 redraws the VLANs originally presented in Figure 7-13 to locate the Root Bridge for both VLANs on Cat-B.
Figure 7-16. Back-to-Back Switches: The Root Bridge for Both VLANs Is on Cat-B
As was the case with Part A of Figure 7-13, the default Configuration BPDUs received on Port 1/1 of Cat-A contains a Port ID of 0x8001, but Port 1/2 receives the value 0x8002. Because 0x8001 is lower, Port 1/1 becames the Root Port for all VLANs by default. However, if you lower VLAN 3's Port Priority to 31 on Cat-B:Port-1/2, it lowers the Port ID that Cat-A:Port-1/2 receives for VLAN 3 to 0x7C01. Because 0x7C01 is less than 0x8001, Cat-A now elects Port 1/2 as the Root Port for VLAN 3 and sends traffic over this link. The syntax to implement this change is as follows:
Cat-B (enable) set spantree portvlanpri 1/2 31 3 Voilá, you have load balancing VLAN 2 is using the left link and VLAN 3 is using the right link.
Tip
Note that the portvlanpri value must be less than the value specified by portpri.
By default, Cat-A is already sending traffic over the 1/1 link, so it is not necessary to add any commands to influence this behavior. However, it is probably a good idea to explicitly put in the command so that you can document your intentions and avoid surprises later:
Cat-B (enable) set spantree portvlanpri 1/1 31 2 This command lowers Cat-B's Port Priority on Port 1/1 to 31 for VLAN 2 and reinforces Cat-A's default behavior of sending traffic over this link for VLAN 2.
Precautions for Using Port/VLAN Priority Load Balancing
Notice that you must adjust the Port Priority on Cat-B to influence the load balancing on Cat-A. Adjusting the Port Priority on Cat-A has no effect as long as Cat-B is the Root Bridge. Recall that the Root Bridge is forwarding on all of its ports and therefore not making any load balancing decisions. This scheme of adjusting one switch's Port Priorities to influence another switch's load balancing is very counterintuitive. Just remember, to use port/VLAN priority load balancing, you must adjust the upstream bridge (the one closer to the Root Bridge, or, in this case, the Root Bridge itself).
Confusion can really set in when you take a look at the output of show spantree when using this form of load balancing. For example, Example 7-1 lists the output for VLAN 3 on Cat-A.
Example 7-1 show spantree Output for VLAN 3 on Cat-A
Cat-A (enable) show spantree 3 VLAN 3 Spanning tree enabled Spanning tree type ieee Designated Root 00-e0-f9-16-28-00 Designated Root Priority 100 Designated Root Cost 19 Designated Root Port 1/2 Root Max Age 20 sec Hello Time 2 sec Forward Delay 15 sec Bridge ID MAC ADDR 00-e0-f9-52-ba-00 Bridge ID Priority 32768 Bridge Max Age 20 sec Hello Time 2 sec Forward Delay 15 sec Port Vlan Port-State Cost Priority Fast-Start Group-method --------- ---- ------------- ----- -------- ---------- ------------ 1/1 1 blocking 19 32 disabled 1/2 1 forwarding 19 32 disabled As you might expect on a device performing load balancing, one port is forwarding and one is blocking (see the last two lines from Example 7-1). You can also observe that the load balancing is having the desired effect because Port 1/2 is the forwarding port. However, notice how the Port Priority is still set at 32. At first this might appear to be a bug. On the contrary, the Port Priority field of show spantree only shows the outbound Port Priority. To see the values Cat-A is receiving, you must look at Cat-B's outbound values as shown in Example 7-2.
Example 7-2 show spantree Output for VLAN 3 on Cat-B
Cat-B (enable) show spantree 3 VLAN 3 Spanning tree enabled Spanning tree type ieee Designated Root 00-e0-f9-16-28-00 Designated Root Priority 100 Designated Root Cost 0 Designated Root Port 1/0 Root Max Age 20 sec Hello Time 2 sec Forward Delay 15 sec Bridge ID MAC ADDR 00-e0-f9-16-28-00 Bridge ID Priority 100 Bridge Max Age 20 sec Hello Time 2 sec Forward Delay 15 sec Port Vlan Port-State Cost Priority Fast-Start Group-method --------- ---- ------------- ----- -------- ---------- ------------ 1/1 1 forwarding 19 32 disabled 1/2 1 forwarding 19 31 disabled There it is Port 1/2's Priority has been set to 31. If you think about the theory behind using set spantree portvlanpri, this makes perfect sense. However, it's very easy to look for the value on Cat-A, the device actually doing the load balancing.
Tip
The received Port ID can be viewed on Cat-A with the show spantree statistics command. This feature is discussed in the "Useful STP Display Commands" section toward the end the chapter.
Keeping all of this straight is especially fun when you are trying to track down some network outage and the pressure is on! In fact, it is this counterintuitive nature of port/VLAN priority that makes it such a hassle to use.
As if the confusing nature of port/VLAN priority wasn't bad enough, set spantree portvlanpri canonly be used in back-to-back situations. Recall that Port ID is evaluated last in the STP decision sequence. Therefore, for it to have any effect on STP's path sections, both Root Path Cost and Sending BID must be identical. Although identical Root Path Costs are fairly common, identical Sending BIDs only occur in one topology: back-to-back bridges. For example, look back at Figure 7-3. Cat-D never receives the same Sending BID on both links because they are connected to completely different switches, Cat-B and Cat-C. Modifying port/VLAN priority in this case has no effect on load balancing.
Tip
Don't use set spantree portvlanpri in the typical model where each IDF switch is connected to redundant MDF switches; it does not work.
Although port/VLAN priority is useful in back-to-back configurations, I recommend that you never use it for that purpose. Why? Because Cisco's EtherChannel technology provides much better performance. Not only does it offer faster failover response, but it often results in better load balancing. EtherChannel is discussed in Chapter 8, "Trunking between Catalysts."
Tip
Don't use set spantree portvlanpri with back-to-back configurations. Instead, use Fast or Gigabit EtherChannel.
Unfortunately, many documents simply present set spantree portvlanpri as the way to do load balancing in a switched network. By failing to mention its limitations and proper use (for example, that it should be coded on the upstream bridge), many users are led on a frustrating journey and never get load balancing to work.
Given all the downsides, why would anyone use port/VLAN priority load balancing? In general, Root Bridge placement and port/VLAN cost provide a much more intuitive, flexible, and easy-to-understand options. However, there are two cases where set spantree portvlanpri might be useful. First, if you are running pre-3.1 NMP code, the port/VLAN cost feature is not available. Second, you might be using back-to-back switches in a configuration where Etherchannel is not an option for example, if you are using non-EtherChannel capable line cards in your Catalyst or the device at the other end of the link is from another vendor (God forbid!).
Note
The version numbers given in this chapter are for Catalyst 5000 NMP images. Currently, this is the same numbering scheme that the Catalyst 4000s and 6000s use. Other Cisco products might use different numbering schemes.
Finally, don't confuse set spantree portvlanpri with the set spantree portpri command. The set spantree portpri command allows you to modify the high-order 6 bits of the Port ID field; however, it modifies these bits for all VLANs on a particular port. This obviously does not support the sort of per-VLAN load balancing being discussed in this section. On the other hand, set spantree portvlanpri allows Port Priority to be adjusted on a per-port and per-VLAN basis.
Note
To save NVRAM storage space, set spantree portvlanpri only allows you to set a total of two priority values for each port. One is the actual value specified on the set spantree portvlanpri command line. The other is controlled through the set spantree portpri command. This behavior can appear very strange to uninformed users. See the section "Load Balancing with Port/VLAN Cost" for more detail (port/VLAN cost also suffers from the same confusing limitation). However, this limitation rarely poses a problem in real-world networks (in general, it is only required for back-to-back switches linked by more than two links, a situation much better handled by EtherChannel).
Potential Issues in Real-World Networks
Before discussing the remaining two load balancing techniques, this section examines issues that arise when trying to load balance in more complex campus networks. In doing so, it demonstrates the potential ineffectiveness of Root Bridge placement and set spantree portvlanpri load balancing as covered in previous sections.
Although Root Bridge placement and set spantree portvlanpri can be effective tools in certain topologies, what if your network consists of two buildings that form a campus as illustrated in Figure 7-17?
Figure 7-17. A Campus Network where Root Bridge Placement and Port Priority Load Balancing Are Not Effective
Figure 7-17 is a simplified diagram of a typical (but not necessarily recommended) campus design. In this design, VLANs 2 and 3 span all switches in both buildings. Each building consists of a pair of redundant MDF (Main Distribution Frame) switches in the main wiring closet. Both MDF switches connect to the IDF (Intermediate Distribution Frame) switches sitting on every floor (only one IDF switch in each building is shown to simplify the diagram). Link costs are also indicated (the IDF links are Fast Ethernet and MDF links are Gigabit Ethernet).
To implement load balancing, you cannot use the Port Priority load balancing technique because the switches are not back-to-back. How about using Root Bridge placement? To load balance in Building 1, you could place the Root Bridge for VLAN 2 on Cat-1A while placing the Root Bridge for VLAN 3 on Cat-1B. This causes VLAN 2's traffic to use the left riser link and VLAN 3's traffic to use the riser on the right. So far so good.
But what does this do to the traffic in Building 2? The IDF switch in Building 2 (Cat-2C) has several paths that it can use to reach the Root Bridge for VLAN 2 (Cat-1A). Which of these paths does it use? Well, refer back to the four-step STP decision criteria covered earlier. The first criterion evaluated is always the Root Bridge. Because everyone is in agreement that Cat-1A is the Root Bridge for VLAN 2, Cat-2C proceeds to the second criterion Root Path Cost. One possibility is to follow the path Cat-2C to Cat-2B to Cat-2A to Cat-1A at a Root Path Cost of 27 (19+4+4). A better option is Cat-2C to Cat-2B to Cat-1A at a cost of 23 (19+4). However, path Cat-2C to Cat-2A to Cat-1A also has a Root Path Cost of 23 (19+4).
Because there are two paths tied for the lowest cost, Cat-2A proceeds to the third decision factor Sending BID. Assume that both Cat-2A and Cat-2B are using the default Bridge Priority (32,768). Also assume that Cat-2A has a MAC address of AA-AA-AA-AA-AA-AA and Cat-2B has a MAC address of BB-BB-BB-BB-BB-BB. Because Cat-2A has the lower BID (32,768.AA-AA-AA-AA-AA-AA), all traffic for VLAN 2 uses the path Cat-2C to Cat-2A to Cat-1B.
OK, how about VLAN 3? Because all switches are in agreement that Cat-1B is the Root Bridge of VLAN 3, Root Path Cost is considered next. One option is to follow the path Cat-2C to Cat-2A to Cat-1A to Cat-1B at a cost of 27 (19+4+4). A better option is Cat-2C to Cat-2A to Cat-1B at a cost of 23 (19+4). However, again, there is an equal cost path along Cat-2C to Cat-2B to Cat-1B (cost =19+4=23). Cat-2C then evaluates the BIDs of Cat-2A and Cat-2B, choosing Cat-2A. VLAN 3 traffic therefore follows the path Cat-2C to Cat-2A to Cat-1B. This does provide load balancing across the campus core, but now both VLANs are using the same IDF riser cable. In other words, the load balancing in Building 1 destroyed the load balancing in Building 2.
Clearly, a new technique is required. Assuming that you want to maintain both VLANs across both buildings (I am using this assumption because it is a common design technique; however, in general, I recommend against it see Chapter 14 for more details), there are two options:
Bridge Priority
Port/VLAN cost
Of these, port/VLAN cost is almost always the better option. However, because set spantree portvlancost was not available until 3.1 Catalyst images, the Bridge Priority technique is discussed first.
Bridge Priority Load Balancing
How can Bridge Priority be used to accomplish load balancing in the two-building campus illustrated in Figure 7-17? As discussed in the previous section, the IDF switch (Cat-2C) found multiple equal cost paths to the Root Bridge. This caused the third decision criterion, Bridge ID, to be evaluated. Because Cat-2A and Cat-2B were using the default Bridge ID values, Cat-2A had the lower BID for all VLANs (32,768.AA-AA-AA-AA-AA-AA versus 32,768.BB-BB-BB-BB-BB-BB). This is precisely what ruined the load balancing in Building 2.
Given this scenario, try lowering the BID on Cat-2B, but just for VLAN 3. For example, you could enter the following command on Cat-2B:
Cat-2B (enable) set spantree priority 1000 3 For consistency, you should also enter the following command on Cat-2A (although this command merely reinforces the default behavior, it helps document your intentions):
Cat-2A (enable) set spantree priority 1000 2 Figure 7-18 illustrates the resulting topology and traffic flows.
Figure 7-18. Load Balancingg Usin Bridge Priority
Note that these commands are adjusting Bridge Priority, not Port Priority. set spantree portvlanpri, the previous technique, was used to adjust Port Priority (the high-order 6 bits of the Port ID field) on a per-port and per-VLAN basis. On the contrary, Bridge Priority is a parameter that is global across all ports on a given switch, but it can be individually set for each VLAN on that switch.
"Wait a minute!" you exclaim. "Isn't this the same value I used to set the Root Bridge?" Yes, it is! The Bridge Priority must be low enough to influence the load balancing of traffic flows, but if it is too low, the bridge wins the Root War and disturbs the entire topology.
Picking the right balance of Port Priorities can be a tricky job in large, complex, and overly flat networks. In fact, it is one of the most significant problems associated with Bridge Priority load balancing. To maintain a consistent pattern, you should use a scheme such as 100, 200, 300,….to position your Root Bridges and 1000, 2000, 3000,…to influence load balancing. This numbering scheme helps you make your network self-documenting while also keeping the load balancing adjustments safely out of the range of the Root Bridge adjustments. This approach assumes that you don't have more than eight backup Root Bridges (if that's not the case, you probably have bigger problems than worrying about load balancing!).
Tip
Use a Bridge Priority numbering scheme that clearly delineates Root Bridge placement from load balancing. For example, use multiples of 100 for your Root Bridges and multiples of 1000 for load balancing.
An additional caveat of using Bridge Priorities to implement load balancing is that it can scale poorly. As your network grows, it is very difficult to keep track of which device has had its Bridge Priority adjusted for each VLAN for which reason (Root Bridge or load balancing).
Furthermore, the technique shares much of the confusion of set spantree portvlanpri. Namely, you must modify MDF switches to implement load balancing on IDF switches. Because it is the received BPDUs that are being evaluated on Cat-2C, changing the BID on Cat-2C has no effect. In short, you must modify the upstream switch when load balancing with either Bridge Priorities or port/VLAN priority.
To make matters even worse, this load balancing scheme creates an awkward situation where one technique is used in Building 1 (Root Bridge placement) and another technique is used in Building 2 (Bridge Priority). It is much more straightforward to use a simple technique that is flexible enough to implement load balancing in both buildings.
Why, then, does anyone choose to use Bridge Priorities to implement load balancing? For one simple reason prior to 3.1 Catalyst 5000 code it was your only choice in certain topologies. However, don't get discouraged starting in 3.1, Cisco offered a feature called port/VLAN cost load balancing that addresses these downsides.
Load Balancing with Port/VLAN Cost
Although Root Bridge placement is attractive because of its simplicity, port/VLAN cost load balancing can be very useful because of its flexibility. By virtue of this versatility, Port/VLAN cost can be an intuitive and effective load balancing scheme for almost every topology.
The port/VLAN priority and Bridge Priority load balancing sections looked at techniques for influencing STP when the costs were equal. The strategy behind port/VLAN cost is based on a very simple and sensible observation: if Root Path Cost is the second STP decision criterion, why are the third and fourth decision criteria being used to implement load balancing? In other words, why not just modify the cost directly?
Catalysts have always supported the set spantree portcost command. However, this command is not useful as a primary technique for load balancing because it changes the cost foreveryVLAN on a given port. It wasn't until 3.1 Catalyst 5000 code (all Catalyst 4000s and 6000s support it) that Cisco added set spantree portvlancost, a command that allows you to tune cost on both a per-port and per-VLAN basis.
Note
Somewhat ironically, the Catalyst 3000s (3000, 3100, and 3200) supported per-port and per-VLAN cost configuration well before the feature was moved into the high-end products such as the Catalyst 5000.
Increasing Path Cost for Load Balancing
Figure 7-19 illustrates how set spantree portvlancost can be used to implement STP load balancing. For simplicity, this example uses Cat-1A as the Root Bridge for all VLANs.
Figure 7-19. Load Balancing Using the set spantree portvlancost Command
First, consider load balancing in Building 1. Cat-1C, the IDF switch in Building 1, has two potential paths to the Root Bridge. It can go directly to Cat-1A over the 1/1 link at a cost of 19, or it can use the 1/2 link to go through Cat-1B at a cost of 23 (19+4). The 1/1 link is fine for VLAN 2, but load balancing requires that VLAN 3 use the 1/2 link. This can be accomplished by increasing the Root Path Cost on the 1/1 link to anything greater than 23 for VLANs that should take the 1/2 link. For example, enter the command from Example 7-3 on Cat-1C to increase the Root Path Cost for VLAN 3 on Port 1/1 to 1000.
Example 7-3 Increasing the Cost for VLAN 3 on Cat-1C:Port-1/1
Cat-1C (enable) set spantree portvlancost 1/1 cost 1000 3 Port 1/1 VLANs 1-2,4-1005 have path cost 19. Port 1/1 VLANs 3 have path cost 1000. Although a comparable entry is not required for VLAN 2 on the 1/2 link, it is generally a good idea to add it for consistency as illustrated in Example 7-4.
Example 7-4 Increasing the Cost for VLAN 2 on Cat-1C:Port-1/2
Cat-1C (enable) set spantree portvlancost 1/2 cost 1000 2 Port 1/2 VLANs 1,3-1005 have path cost 19. Port 1/2 VLANs 2 have path cost 1000. The previous two commands increase the Root Path Cost on the undesirable link high enough to force traffic across the other IDF-to-MDF link. For example, the command in Example 7-3 discourages VLAN 3's traffic from using the 1/1 link, causing it to use the 1/2 link. However, if either riser link fails, all of the traffic rolls over to the remaining connection.
As mentioned earlier, it is important to understand the difference between Root Path Cost and Path Cost. This point is especially true when working with port/VLAN cost load balancing. Root Path Cost is the cumulative Spanning Tree cost from a bridge or switch to the Root Bridge. Path Cost is the amount that is added to Root Path Cost as BPDUs are received on a port. Notice that set spantree portvlancost is manipulating Path Cost, not Root Path Cost. It might help if you remember the command as set spantree portvlan pathcost (just don't try typing that in!).
Decreasing Path Cost for Load Balancing
An alternate approach is to decrease the cost on the desirable link. In fact, this is actually the manner in which the set spantree portvlancost command syntax was designed to support. Here is the full syntax of the set spantree portvlancost command:
set spantree portvlancost mod_num/port_num [cost cost_value] [preferred_vlans]
Notice that the set spantree portvlancost command allows you to omit several of the parameters as a shortcut. If you omit the cost parameter, it lowers the cost by one from its current value. If you omit the preferred_vlans parameter, it uses the VLAN list from the last time the command was used. In other words, the command in Example 7-5 is designed to make Port 1/2 the preferred path for VLAN 3.
Example 7-5 Selecting Cat-1C:Port-1/2 As the Preferred Path for VLAN 3 Using the Automatically Calculated Value
Cat-1C> (enable) set spantree portvlancost 1/2 3 Port 1/2 VLANs 1-2,4-1005 have path cost 19. Port 1/2 VLANs 3 have path cost 18. Tip
Unlike most Spanning Tree commands on Catalysts that substitute the default value of 1 when the vlan parameter is omitted, set spantreee portvlancost uses the same VLAN (or VLANs) as the previous use of this command. To avoid surprises, it is safer to always specify both the cost and the preferred_vlansparameters.
However, lowering the cost to 18 on Port 1/2 for VLAN 3 does not work in situations such as Cat-1C in Figure 7-19. In this case, Cat-1C sees two paths to the Root Bridge. As explained earlier, the Root Path Cost values before tuning are 19 on the 1/1 link and 23 (19+4) on the 1/2 link. Lowering the 1/2 Path Cost by one results in a Root Path Cost for Port 1/2 of 22, not enough to win the Root Port election.
One solution is to manually specify a cost parameter that is low enough to do the trick as in Example 7-6.
Example 7-6 Selecting Cat-1C:Port-1/2 As the Preferred Path for VLAN 3 By Manually Specifying a Lower Cost on Port 1/2
Console> (enable) set spantree portvlancost 1/2 cost 14 3 Port 1/2 VLANs 1-2,4-1005 have path cost 19. Port 1/2 VLANs 3 have path cost 14. This lowers the cumulative Root Path Cost on Port 1/2 to 18 (14+4) and causes it to win out against the cost of 19 on Port 1/1.
However, this approach might not be stable in the long run. What if the link between Cat-1A and Cat-1B is replaced with Fast Ethernet or Fast EtherChannel? Or what if an additional switch is added in the middle of this link? In fact, if anything is done to increase the cost between Cat-1A and Cat-1B, this load balancing scheme fails. Therefore, as a general rule of thumb, it is better to increase the cost of undesirable paths than decrease the cost of desirable paths.
Tip
Increasing the cost of undesirable paths is more flexible and scalable than decreasing the cost of desirable paths.
Advantages of Port/VLAN Cost Load Balancing
Getting back to the example in Figure 7-19, now consider the load balancing for Building 2. First, force STP to use the Cat-2C:Port-1/2 link for VLAN 3 by entering the commands in Example 7-7 and 7-8 on Cat-2C.
Example 7-7 Selecting Cat-2C:Port-1/2 As the Preferred Path for VLAN 3 By Manually Specifying a Higher Cost on Port 1/1
Cat-2C (enable) set spantree portvlancost 1/1 cost 1000 3 Port 1/1 VLANs 1-2,4-1005 have path cost 19. Port 1/1 VLANs 3 have path cost 1000. Then, force VLAN 2 over the 1/1 link, as shown in Example 7-8.
Example 7-8 Selecting Cat-2C:Port-1/1 As the Preferred Path for VLAN 2
Cat-2C (enable) set spantree portvlancost 1/2 cost 1000 2 Port 1/2 VLANs 1,3-1005 have path cost 19. Port 1/2 VLANs 2 have path cost 1000. See how much easier set spantree portvlancost is to use than the port/VLAN priority and Bridge Priority load balancing? First, it is much easier to visualize the impact that the commands are having on the network. Second, both IDF switches use similar and consistent commands. Third, and best of all, the commands are entered on the very switch where the load balancing is occurring the IDF switch. If you have an IDF switch with multiple uplinks, just Telnet to that device and use set spantree portvlancost to spread the load over all available links. With port/VLAN cost, there is no need to make strange modifications to upstream switches.
The results of using set spantree portvlancost are also much easier to read in show spantree. For example, the output in Example 7-9 would appear for VLAN 2 on Cat-2C, the IDF switch in Building 2.
Example 7-9 show spantree Output for VLAN 2 After Using Port/VLAN Cost Load Balancing
Cat-2C (enable) show spantree 2 VLAN 2 Spanning tree enabled Spanning tree type ieee Designated Root 00-e0-f9-16-28-00 Designated Root Priority 100 Designated Root Cost 23 Designated Root Port 1/1 Root Max Age 20 sec Hello Time 2 sec Forward Delay 15 sec Bridge ID MAC ADDR 00-e0-f9-F2-44-00 Bridge ID Priority 32768 Bridge Max Age 20 sec Hello Time 2 sec Forward Delay 15 sec Port Vlan Port-State Cost Priority Fast-Start Group-method --------- ---- ------------- ----- -------- ---------- ------------ 1/1 1 forwarding 19 32 disabled 1/2 1 blocking 1000 32 disabled The increased cost on the 1/2 link is plainly shown along with the fact that Port 1/1 is in the Forwarding state. Also notice that Port 1/1 is acting as the Root Port.
The output in Example 7-10 shows the Spanning Tree information for VLAN 3 on the same switch.
Example 7-10 show spantree Output for VLAN 3 After Using Port/VLAN Cost Load Balancing
Cat-2C (enable) show spantree 3 VLAN 3 Spanning tree enabled Spanning tree type ieee Designated Root 00-e0-f9-16-28-00 Designated Root Priority 100 Designated Root Cost 23 Designated Root Port 1/2 Root Max Age 20 sec Hello Time 2 sec Forward Delay 15 sec Bridge ID MAC ADDR 00-e0-f9-F2-44-00 Bridge ID Priority 32768 Bridge Max Age 20 sec Hello Time 2 sec Forward Delay 15 sec Port Vlan Port-State Cost Priority Fast-Start Group-method --------- ---- ------------- ----- -------- ---------- ------------ 1/1 1 blocking 1000 32 disabled 1/2 1 forwarding 19 32 disabled In the case of Example 7-10, the 1/1 port is Blocking, whereas the 1/2 port is Forwarding. The Root Port has also shifted to 1/2 with a Root Path Cost of 23 (19 for the riser link plus 4 to cross the Gigabit Ethernet link between Cat-2B and Cat-1A).
Tip
Carefully observe the difference between Root Path Cost and Path Cost. Root Path Cost is the cumulative cost to the Root Bridge. Path Cost is the amount that each port contributes to Root Path Cost.
Because it is both flexible and easy to understand, port/VLAN cost is one of the most useful STP load balancing tools (along with Root Bridge placement). The only requirement is that you run 3.1 or higher code on the switches where you want to implement the load balancing (generally your IDF switches). Note that you do not need to run 3.1+ code everywhere, it is only necessary on the actual devices where set spantree portvlancostload balancing is configured (modifying the set spantree portvlancost on some devices does not create any interoperability problems with your other switches).
Precautions for Using Port/VLAN Cost Load Balancing
However, there is one potentially confusing matter that you should keep in mind when using set spantree portvlancost: each port is only allowed two Port Cost values (very similar to how each port is only allowed two port/VLAN priority values). The first value is the default cost for the port. This value is derived from the bandwidth chart presented in Table 6-1 of Chapter 6 and can be modified with the set spantree portcost command (it modifies cost for every VLAN on a port). The second value is the one configured with set spantree portvlancost. However, only a single set spantree portvlancost value can exist on a given port. If you configure multiple set spantree portvlancost values on a single port, all of the VLANs ever configured with set spantree portvlancost adopt the most recent value.
Tip
As with port/VLAN priority, the per-VLAN value must be less than the per-port value. In other words, the set spantree portvlancost value must be less than the set spantree portcost value.
For example, the command in Example 7-11 increases the cost to 1000 for VLANs 2 and 3 on Port 1/1.
Example 7-11 Increasing Port/VLAN Cost on Port 1/1 for VLANs 2 and 3
Cat-A> (enable) set spantree portvlancost 1/1 cost 1000 2-3 Port 1/1 VLANs 1,4-1005 have path cost 19. Port 1/1 VLANs 2-3 have path cost 1000. Next, try increasing the Path Cost for VLAN 5 to 2000 as in Example 7-12.
Example 7-12 Trying to Specify a Different Port/VLAN Cost Value
Console> (enable) set spantree portvlancost 1/1 cost 2000 5 Port 1/1 VLANs 1,4,6-1005 have path cost 19. Port 1/1 VLANs 2-3,5 have path cost 2000. Notice how it also changes the Path Cost for VLANs 2 and 3. A quick look at the output of show spantree in Example 7-13 confirms the change.
Example 7-13 Only the Most Recently Specified Port/VLAN Cost Value Is Used
Console> (enable) show spantree 1/1 Port Vlan Port-State Cost Priority Fast-Start Group-Method --------- ---- ------------- ----- -------- ---------- ------------ 1/1 1 forwarding 19 31 disabled 1/1 2 forwarding 2000 31 disabled 1/1 3 forwarding 2000 31 disabled 1/1 4 forwarding 19 31 disabled 1/1 5 forwarding 2000 31 disabled 1/1 6 forwarding 19 31 disabled 1/1 7 forwarding 19 31 disabled 1/1 8 forwarding 19 31 disabled 1/1 9 forwarding 19 31 disabled 1/1 10 forwarding 19 31 disabled
Poof! The cost of 1000 is gone. As mentioned in the "Port/VLAN Priority Load Balancing" section that covered the set spantree portvlanpri command (which also exhibits the behavior seen in Example 7-13), this sleight of hand is a subtle side effect caused by a technique the Catalysts use to save NVRAM storage. Internally, Catalysts store three different values related to cost per port:
A global cost for that port
A single set spantree portvlancost value
A list of VLANs using the set spantree portvlancost value
Therefore, separate cost values cannot be stored for every VLAN. However, the good news is that, apart from its strange appearances, this is not a significant drawback in most situations. Networks with lots of redundancy and multiple links might find it a limitation, but most never notice it at all.
Spanning Tree Load Balancing Summary
In review, there are four techniques available for Spanning Tree load balancing on Catalyst equipment:
Root Bridge Placement Traffic flows can be influenced by carefully locating Root Bridges throughout your network. This option is extremely viable and useful for organizations utilizing the multilayer design model discussed in Chapter 14. However, for networks employing less hierarchical designs, it can be hopeless.
Port/VLAN Priority set spantree portvlanpri can be used to load balance between back-to-back switches. This technique allows network administrators to modify Port ID values on upstream switches to influence forwarding and blocking decisions on neighboring downstream switches. For a variety of reasons, the option is rarely useful.
Bridge Priority Bridge Priorities can be used. In general, this technique is only useful in older software images (pre-3.1) where set spantree portvlancost is not available. As with set spantree portvlanpri, it requires that load balancing parameters be entered on the switch that is upstream of the switch actually performing the load balancing.
Port/VLAN Cost set spantree portvlancost can be utilized. This technique is the most flexible option available for STP load balancing. In flat networks, it is also the simplest to implement. The only requirement is that you use 3.1 or later code on your devices. It allows the command to be entered and observed on the actual switch where the load balancing is being performed.
In short, Spanning Tree load balancing almost always boils down to one of two options. First, when using the MLS form of Layer 3 switching to build networks according to the multilayer design model (see Chapters 11 and 14), the Root Bridge Placement form of load balancing is generally most appropriate. Second, port/VLAN cost load balancing should be used in almost all other situations.
EAN: N/A
Pages: 223