Scenario 3-4: Configuring EtherChannel

In this scenario you learn how to configure an EtherChannel bundle between two Cisco Catalyst switches. Figure 3-16 illustrates the topology used for this scenario, which extends the topology used for the previous scenarios.

Figure 3-16. Scenario 3-4 Topology

In Figure 3-16, the trunk between Switch-A and Switch-B is to operate over two physical interfaces configured as an EtherChannel bundle, rather than the single physical interface used in previous scenarios. This configuration increases the bandwidth of the connection between Switch-A and Switch-B and also protects against a single circuit or interface failure.

Understanding EtherChannel

Before configuring EtherChannel, you must understand how EtherChannel bundles negotiate and understand the load sharing mechanisms supported by your Cisco Catalyst switches. The following sections are now presented:

• PAgP negotiation

• Load sharing

PAgP Negotiation

Just as trunks use a negotiation protocol (DTP) between Cisco switches to determine whether or not a trunk should form, so to does EtherChannel, which uses PAgP as the protocol for negotiating an EtherChannel bundle.

It is important to understand which PAgP modes co-exist and form an EtherChannel. Table 3-5 describes all of the possible combinations of PAgP modes that can be configured between two directly connected PAgP peers, assuming the names of the peers are Switch-A and Switch-B.

Table 3-5. PAgP Negotiation Matrix

		Switch-B
	PAgP Mode	On	Off	Auto	Desirable
Switch-A	On	Channel	Not Channel (ErrDisable)	Not Channel (ErrDisable)	Not Channel (ErrDisable)
	Off	Not Channel (ErrDisable)	Not Channel	Not Channel	Not Channel
	Auto	Not Channel (ErrDisable)	Not Channel	Not Channel	Channel
	Desirable	Not Channel (ErrDisable)	Not Channel	Channel	Channel

Notice that there are only three combinations of PAgP modes that will successfully channel (form a bundle):

• Desirable and desirable Forms because both parties actively attempt to negotiate a channel. This combination is the recommended configuration.

• Desirable and auto Forms because one side actively attempts to negotiate, while the other side responds only to PAgP negotiation.

• On and on Always forms a channel because both parties are hard coded; no PAgP frames are sent.

It is very tempting to use a PAgP mode of on because normally a bundle is static and does not change. Therefore, why would you need to negotiate the bundle? However, if you decide to use a PAgP mode of on, notice in Table 3-5 that forcing a bundle to use a on mode results in the bundle not channeling if the PAgP mode of the other side is set to anything other than a mode of on as well. Notice also that an ErrDisable condition also occurs, which is caused by the on mode forcing an EtherChannel bundle to be always up while the other side of the connection is not configured as a bundle. This configuration can cause spanning-tree loops in the network. Cisco Catalyst switches detect and immediately shut down the looped ports, placing them into an ErrDisable state, which indicates a serious error caused the switch to shut down the ports. The following shows an example of the console message displayed on Switch-B when a spanning-tree loop is detected due to incorrect PAgP configuration:

2002 Oct 31 16:58:23 %SPANTREE-2-CHNMISCFG: STP loop

 - channel 2/1-2 is disabled in vlan/instance 1 

NOTE

Although the ErrDisable mechanism protects networks based upon Cisco Catalyst switch, occurrences of the ErrDisable are highly undesirable, indicating a serious problem with the network, and should be always avoided.

Even if both sides of a bundle are configured with a PAgP mode of on (which means a bundle forms), it is important to be aware that the redundancy of the link is compromised in the event of a failure on the link that is carrying spanning-tree traffic (see Chapter 4 for more detail on spanning tree). Basically, this means that spanning tree declares an entire bundle down if the link that carries spanning-tree traffic fails. Clearly, this result is undesirable and negates the redundancy benefits of EtherChannel. If, however, you configure a mode of desirable, each switch ensures that spanning-tree traffic is forwarded over the redundant link, ensuring the spanning-tree topology remains stable during a physical interface failure.

For all of the reasons described so far, it is recommended you use a mode of desirable on each end because configuring this mode ensures that an ErrDisable state is never generated (unless the remote side is configured with a PAgP mode of on) and also ensures the link failure scenario described does not affect the spanning-tree topology.

Of course if you are connecting a Cisco Catalyst switch to a device that does not support PAgP, such as a server or router, you need to configure a PAgP mode of on to ensure the bundle forms with the remote devices.

PAgP Negotiation Delays

If you configure a PAgP mode of auto or desirable, when a port first initializes due to a physical link being detected, the Layer 2 line protocol of the port does not come up until PAgP negotiates a bundle or until PAgP negotiation times out. If a bundle is successfully negotiated, this port coming up normally occurs within a matter of seconds, but if a PAgP negotiation timeout occurs, a port does not come up at Layer 2 for approximately 15-20 seconds. In other words, if a non-EtherChannel capable device connects to a switch port that has a PAgP mode of auto or desirable configured, it does get a Layer 2 connection for approximately 15-20 seconds. If workstations are connected, this situation can cause unacceptable delays when a workstation starts up, especially if the workstations are fast computers that boot up quickly. The workstation might not be able to obtain an IP address via Dynamic Host Configuration Protocol (DHCP), causing logon failures to the network among other problems.

TIP

On CatOS switches, a PAgP mode of auto is configured by default on EtherChannel capable ports, meaning by default all non-EtherChannel devices experience an unnecessary 18-19 second delay upon port initialization. This delay adds to other possible delays caused by DTP negotiation and spanning-tree initialization once the port is handed to spanning tree by PAgP, causing delays of up to a minute in extreme cases. To minimize port initialization delays, always configure a PAgP mode of off on ports that are connected to non-EtherChannel devices, such as workstations. On Cisco IOS switches, the default PAgP mode of ports is off, so the default configuration does not suffer this problem.

PAgP Silent and Non-Silent Operation

When a PAgP mode of auto or desirable is configured on a port, you also have the option of configuring either of the following operational parameters:

• Non-Silent An EtherChannel bundle does not form, until bidirectional connectivity has been confirmed. This parameter means that ports in the EtherChannel bundle must receive data, as well as be able to transmit data, in order for the bundle to form. Non-silent is default on Catalyst 5000 fiber-based FastEthernet and fiber-based gigabit Ethernet ports. This mode protects against unidirectional failures, where one of the transmit or receive links may fail. These failures are common on fiber-based connections and can cause bridging loops.

• Silent An EtherChannel bundle forms, even if data has not been received from the remote device. This mode is the default mode of operation on all Catalyst 4000, 6000, and Cisco IOS-based switch ports, as well as on Catalyst 5000 copper ports. For unidirectional failures on Cisco IOS-based Catalyst ports, Catalyst 4000 ports, and Catalyst 6000 ports, other protocols, such as unidirectional link detection (UDLD), which detect these failures much more quickly than non-silent PAgP operation, are used to protect against unidirectional failures.

Cisco recommends that you do not modify the default silent or non-silent mode of operation.

Load Sharing

If you are implementing EtherChannel for performance benefits, it is important that you understand that the load-sharing mechanism used by a bundle affects performance. Depending on the Cisco Catalyst switch platform, load sharing is supported based on Layer 2, Layer 3, and Layer 4 source and/or destination address/port values.

Table 3-6 shows the supported load sharing mechanisms based upon Catalyst platform.

Table 3-6. Cisco Catalyst EtherChannel Load Sharing Capabilities

Platform	Load Sharing Mechanisms
2900XL/3500XL 2950/3550 (Cisco IOS)	Source MAC Destination MAC
2900/4000/5000/5500 Early Catalyst 6000 Supervisor 1[1] (CatOS)	Source MAC XOR Destination MAC (Non-configurable)
Catalyst 6000/6500 Supervisor 1a with PFC/PFC2 (CatOS and Cisco IOS)	Source MAC Destination MAC Source MAC and Destination MAC Source IP Destination IP Source IP and Destination IP (default)
Catalyst 6000 Supervisor 2 with PFC2 (CatOS and Cisco IOS) and Catalyst 4000 Supervisor 3/4 (Cisco IOS)	Source MAC Destination MAC Source MAC and Destination MAC Source IP Destination IP Source IP and Destination IP (default) Source Port Destination Port Source Port and Destination Port

^[1] Applies to early implementations of the Catalyst 6000 Supervisor 1. Issue the show module command, and if the sub-type is listed as "L2 Switching Engine I WS-F6020," then only a non-configurable distribution method based upon source and destination MAC addresses is supported.

The load sharing mechanism is particularly important when you provide connectivity to a small amount of devices via a bundle. In this situation, large amounts of traffic are being sent to and from a single or a few devices, which can overload a single link (refer Figure 3-10). Make sure you understand the major traffic flows between devices in your network to ensure your load sharing mechanism is adequate for your environment.

EtherChannel Support on Cisco Catalyst Switches

Most Cisco Catalyst switches support EtherChannel; however, it is important to understand that how EtherChannel is supported can vary from platform to platform. The following lists some important platform considerations you should well understand before implementing EtherChannel:

• What is the maximum number of EtherChannel bundles required?

• What is the maximum number of ports per bundle required?

• Are contiguous ports required?

• Are there specific restrictions on the collection of ports you can configure in a bundle?

• Do all ports need to be on the same switching module?

• Are there any load sharing considerations? (see Table 3-6)

The limitations of what you can or can't do are governed by the switch port hardware and software you are using. Table 3-7 shows the limitations of the various Catalyst platforms, assuming the hardware is EtherChannel capable.

Table 3-7. EtherChannel Capabilities

Platform	Maximum Number of Channels	Maximum Number of Ports per Channel	Contiguous Ports?	Specific collection of ports?	Same Module?
6000/6500 (Cisco IOS)	64 256[1]	8	No	No	No
6000/6500 (CatOS)	128	8	No	No	No
5000/5500[2] (CatOS)	-	4[3]	Yes[3] (2 or 4)	Yes[4]	Yes
4000 (Cisco IOS)	64	8	No	No	No
2900/4000 (CatOS)	126	8	No	No	No
3550	64	8	No	No	n/a
2950	6	8	No	No	n/a
2900XL/3500XL	12	8 (source-base) Unlimited (destination-based)	No	No	n/a

^[1] Catalyst 6000/6500 Native IOS 12.1(2)E and earlier.

^[2] Supported only on modules with an Ethernet Bundling Controller (EBC) onboard. These modules include the Supervisor 2/3 engines and some line cards.

^[3] On the Catalyst 5000 family gigabit EtherChannel module (WS-X5010), an EtherChannel bundle can consist of any two to eight ports on the module. Ports in an EtherChannel do not have to be contiguous.

^[4] For modules that support only a maximum of four ports, an EBC is allocated to each group of four ports. The ports that form a bundle must be managed by the same EBC, and bundles must use ports that start from the first port in the EBC group or ports. An exception to the latter is the WS-X5225R module, which allows you to configure bundles that do not start at the first port within a group (e.g., group = 2/1-4, bundle can be configured on 2/3-4).

As you can see from Table 3-7, each of the switch platforms has varying capabilities. The platform that has the most restrictions is the Catalyst 5000/5500 family, where most modules require EtherChannel to be configured on contiguous ports and only in certain configurations. An EtherChannel controller typically exists for every four ports on a module. For example, you might be able to configure a four-port bundle only on ports 2/1-4, 2/5-8, 2/9-12 and so on (you cannot configure a bundle from 2/3-2/6). If you want to configure a two-port bundle, you must start from the beginning of each range for a particular EtherChannel controller. For example, ports 2/1-2 are okay, but ports 2/3-4 are not OK because they do not start at the beginning of a range. Modern EtherChannel implementations do not have such strict limitations as the older Catalyst 5000/5500 family.

NOTE

You can use the show port capabilities command on CatOS to determine supported EtherChannel configurations. This command is demonstrated later in this scenario.

Configuration Tasks

Before you configure EtherChannel, you must ensure that you are aware of any restrictions that the hardware you are configuring might have. Refer to Table 3-7 for information on how EtherChannel might be configured on the various Cisco Catalyst platforms, and refer to Table 3-6 for information on the load sharing mechanisms that are supported.

Configuration of EtherChannel consists of the following tasks:

• Configuring physical port parameters

• Creating an EtherChannel bundle

• Configuring EtherChannel load distribution

• Verifying EtherChannel configuration

Configuring Physical Port Parameters

A very important prerequisite of configuring EtherChannel bundles is that each of the physical ports or interfaces that make up the EtherChannel bundle must be configured identically to ensure the bundle comes up and operates correctly. These physical port/interface parameters include the following:

• Port speed/duplex settings Ensure all ports operate at the same speed and duplex setting.

• VLAN configuration Ensure all ports belong to the same VLAN.

• Trunking configuration Ensure trunking modes are identical and that the allowed VLANs for each trunk are the same. For 802.1Q trunks, ensure that the native VLAN for each trunk is identical (trunking is covered later in this chapter).

• Spanning tree configuration Ensure path cost, port priority, and PortFast settings are identical (spanning tree is covered in Chapter 4).

• SPAN destination ports You cannot configure an EtherChannel bundle that includes SPAN destination ports (SPAN is covered in Chapter 9).

• Secure ports You cannot configure an EtherChannel bundle that includes secure ports (secure ports are covered in Chapter 7, "Multicast Routing and Switching").

• Dynamic VLAN ports Do not configure EtherChannel ports as dynamic VLAN or 802.1x ports. Doing so can adversely affect switch performance (dynamic VLANs and 802.1x are covered in Chapter 7).

• Protocol filtering EtherChannel bundles do not form if protocol filtering is configured differently on the ports (protocol filtering is covered in Chapter 7).

• Quality of Service (QoS) configuration All ports must have identical QoS configurations; otherwise, an EtherChannel bundle does not form (QoS is covered in Chapter 8, "Traffic Filtering and Security").

• Jumbo frames Ports with different jumbo frame configurations do not form a bundle.

As you can see, many configuration parameters must be set identically; otherwise, the switch operating system (CatOS or Cisco IOS) rejects the configuration.

Just as it is important to configure all ports on one side of an EtherChannel bundle identically, it is also very important that the ports on each switch that form each side of the bundle are also configured identically. In other words, you should ensure that the requirements just discussed are also applied to the remote switch ports that make up the other side of the EtherChannel bundle. Example 3-22 demonstrates configuring interfaces on Switch-A in Figure 3-16 to ensure that the interfaces form an EtherChannel bundle.

Example 3-109. Configuring the Physical Interfaces of an EtherChannel Bundle on Cisco IOS

 Switch-A# configure terminal Switch-A(config)# interface range FastEthernet0/1 - 2 Switch-A(config-if-range)# speed 100 Switch-A(config-if-range)# duplex full Switch-A(config-if-range)# switchport trunk encapsulation dot1q Switch-A(config-if-range)# switchport mode dynamic desirable Switch-A(config-if-range)# switchport trunk native vlan 10 Switch-A(config-if-range)# switchport trunk allowed vlan 2-5,10,1002-1005 Switch-A(config-if-range)# switchport trunk pruning vlan 2-4 

In Example 3-22, the configuration of both physical interfaces must be identical; hence, the trunk configurations applied in Scenarios 3-2 and 3-3 are applied to both interfaces to ensure the configurations on both are identical. The remote ports on Switch-B should also be configured with matching settings. Example 3-23 demonstrates configuring ports on Switch-B in Figure 3-16 to ensure that each port has the same configuration as other local ports and the remote interfaces on Switch-A.

Example 3-110. Configuring the Physical Interfaces of an EtherChannel Bundle on CatOS

 Switch-B> (enable) set port speed 2/1-2 100 Ports 2/1-2 transmission speed set to 100Mbps. Switch-B> (enable) set port duplex 2/1-2 full Ports 2/1-2 set to full-duplex. Switch-B> (enable) set trunk 2/1 desirable dot1q Port(s)  2/1 trunk mode set to desirable. Port(s)  2/1 trunk type set to dot1q. Switch-B> (enable) set trunk 2/2 desirable dot1q Port(s)  2/2 trunk mode set to desirable. Port(s)  2/2 trunk type set to dot1q. Switch-B> (enable) set vlan 10 2/1-2 VLAN 10 modified. VLAN 1 modified. VLAN  Mod/Ports ---- ----------------------- 10    2/1-2 Switch-B> (enable) clear trunk 2/1 2-1005 Removing Vlan(s) 2-1005 from allowed list. Port  2/1 allowed vlans modified to 1. Switch-B> (enable) clear trunk 2/2 2-1005 Removing Vlan(s) 2-1005,1025-4094 from allowed list. Port  2/2 allowed vlans modified to 1. Switch-B> (enable) set trunk 2/1 2-5,10 Adding vlans 2-5,10 to allowed list. Port(s)  2/1 allowed vlans modified to 2-5,10. Switch-B> (enable) set trunk 2/2 2-5,10 Adding vlans 2-5,10 to allowed list. Port(s)  2/2 allowed vlans modified to 2-5,10. 

NOTE

If you are unsure of the current configuration of an interface or port, use the show interface command on Cisco IOS or the show port mod/port command on CatOS.

Configuring an EtherChannel Bundle

After ensuring that the settings for each of the physical interfaces/ports that make up an EtherChannel bundle are compatible with EtherChannel and are configured identically across all local interfaces/ports and remote interfaces/ports, you can create EtherChannel bundles.

Cisco IOS Configuration

On Cisco IOS, an EtherChannel bundle is referred to as a channel group and is represented as a single logical interface known as a port channel interface. When configuring Layer 2 EtherChannel bundles (i.e., Layer 2 switch ports make up the bundle), you don't need to explicitly create the appropriate port channel interface for an EtherChannel bundle. Instead, the channel-group interface configuration mode command is used to assign the physical interface to an EtherChannel bundle, which automatically creates a new port channel interface if one does not already exist. The channel-group interface configuration mode has the following syntax:

 Switch(config-if)# channel-group channel-group-number mode {auto | desirable   [non-silent] | on} 

The channel-group-number parameter defines the channel group number that is assigned to the bundle created. Valid values for this parameter range from 1 up to the maximum number of EtherChannel bundles supported on the switch (see Table 3-7). This number is also used to identify the logical port channel interface. The mode keyword allows you to specify the appropriate auto, desirable, or on keywords, which define the PAgP mode of operation. It is recommended that you always configure a mode of desirable because this mode ensures that interface failures do not cause issues with spanning tree. You also normally do not need to modify the silent or non-silent mode of operation (by default, the silent mode is configured on Cisco IOS switches). Example 3-24 demonstrates configuring an EtherChannel bundle on Switch-A in Figure 3-16.

Example 3-111. Creating an EtherChannel Bundle on Cisco IOS

 Switch-A# configure terminal Switch-A(config)# interface range FastEthernet0/1 - 2 Switch-A(config-if-range)# channel-group 1 mode desirable Switch-A(config-if-range)# end 

The configuration of Example 3-24 automatically assigns interfaces FastEthernet0/1 and FastEthernet0/2 to channel group 1 and configures the interfaces to use the desirable mode for PAgP. If a new channel group has been created by the interface configuration (as is the case in Example 3-24), a new logical port channel interface is also created.

To verify that you have created an EtherChannel bundle, use the show etherchannel summary command, as demonstrated in Example 3-25.

Example 3-112. Verifying EtherChannel Configuration

 Switch-A# show etherchannel summary Flags:  D - down        P - in port-channel         I - stand-alone s - suspended         R - Layer3      S - Layer2         u - unsuitable for bundling         U - port-channel in use         d - default port Group Port-channel  Ports -----+------------+----------------------------------------------------------- 1     Po1(SU)     Fa0/1(P)   Fa0/2(P) 

In Example 3-25, you can see the channel group #1, and that a port-channel interface has been created called Po1. The physical interfaces that comprise the group are listed in the Ports section. Notice the flags indicate that channel group #1 is a Layer 2 EtherChannel bundle (indicated by the S flag) and that each physical port is currently operating in the bundle (indicated by the P flag).

NOTE

Cisco Catalyst Layer 3 switches also support the configuration of Layer 3 EtherChannel bundles, which are equivalent to an EtherChannel bundle on a Cisco router. This feature is discussed in Chapter 5, "Inter-VLAN Routing."

Although Switch-B has not yet been configured for EtherChannel, notice that an EtherChannel bundle has formed. By default CatOS switches operate in a PAgP mode of auto, which means that an EtherChannel bundle forms (see Table 3-5). Cisco IOS switches in contrast have a PAgP mode of off by default, requiring explicit configuration.

CatOS Configuration

On CatOS, an EtherChannel bundle is also referred to as a channel group and is formed based upon a logical entity known as an administrative group. A channel group represents an actual set of physical ports that currently form an active bundle. An administrative group on the other hand defines the list of available ports that a channel group may consist of. For example, an administrative group might specify a list of ports, say 2/1-2/4. Assume that a channel group is formed that initially includes each of these ports. Next consider what happens when a port (say port 2/3) fails. From a physical point of view, traffic must not be sent over the failed port; hence, the channel group needs to be updated to exclude the failed port (i.e., the channel group consists only of ports 2/1, 2/2, and 2/4). However, from a configuration point of view, the administrative group still needs to include ports 2/1-2/4 to ensure the channel group adds the failed port after that port has been restored.

Each channel group and administrative group is represented by a numeric identifier. With CatOS, you don't actually specify a channel group ID like you do with Cisco IOS. CatOS takes care of this automatically for you and then creates an administrative group. If you compare administrative groups with Cisco IOS, the equivalent Cisco IOS entity is a port channel interface. In Cisco IOS, the port channel interface ID is the same as the channel group ID; with CatOS, however, the administrative group ID is not the same as the channel group ID. However, at the end of the day, this numbering is essentially transparent to administrators, so if you find this scheme a little confusing, don't worry too much.

CatOS is also different to Cisco IOS in that any EtherChannel capable port is actually configured with a PAgP mode of auto by default. This default means that you don't actually necessarily need to configure anything for an EtherChannel bundle to form on a CatOS switch as long as the remote switch has ports configured with a PAgP mode of on or desirable. This characteristic explains why in Example 3-25 you saw that the EtherChannel bundle on Switch-A was up, without any configuration of Switch-B.

NOTE

Don't get lazy and forget to explicitly configure your EtherChannel bundles. Remember that the recommended EtherChannel PAgP mode to ensure physical interface failures do not affect spanning tree is desirable.

Before configuring EtherChannel on a CatOS switch, it is a good idea that you verify that the ports you wish to configure in an EtherChannel bundle do actually support EtherChannel. You can use the show port capabilities command to verify whether or not a port supports EtherChannel. Example 3-26 demonstrates the use of this command on Switch-B in Figure 3-16.

Example 3-113. Verifying a Port Supports EtherChannel on CatOS

 Switch-B> (enable) show port capabilities 2/1 Model                    WS-X6148-RJ45V Port                     2/1 Type                     10/100BaseTX Speed                    auto,10,100 Duplex                   half,full Trunk encap type         ISL,802.1Q Trunk mode               on,off,desirable,auto,nonegotiate Channel                  2/1-48 Flow control             no Security                 yes Dot1x                    yes Membership               static,dynamic ... <output truncated> ... 

In Example 3-26, the shaded line indicates that EtherChannel is supported on all ports of the module.

On CatOS by default, a set of administrative groups are already pre-configured, which can be displayed by using the show channel group command. Example 3-27 demonstrates the use of this command on Switch-B before any custom EtherChannel configuration has been applied.

Example 3-114. Displaying Administrative Groups on CatOS

 Switch-B> (enable) show channel group Admin Group  Ports -----------  ----------------------------------------------- 1            1/1-2 2            2/1-4 3            2/5-8 4            2/9-12 5            2/13-16 6            2/17-20 7            2/21-24 8            2/25-28 9            2/29-32 10           2/33-36 11           2/37-40 12           2/41-44 13           2/45-48 

In Example 3-27, you can see that on Switch-B each administrative group consists of four physical ports by default (except for the two gigabit Ethernet ports on the Supervisor). It is important that you understand the default grouping because it defines how the switch forms EtherChannel bundles by default. For example, in Example 3-25 you saw that Switch-B automatically formed an EtherChannel bundle on ports 2/1 and 2/2 with Switch-A. Referring to Example 3-27, ports 2/1 and 2/2 fall within an administrative group (#2); hence, a channel group can be formed as both ports are within the same administrative group. If, however, Switch-B were connected to Switch-A via ports 2/4 and 2/5, you can see that each port would be in separate administrative groups (port 2/4 is in group #2 and port 2/5 is in group #3). Because both ports would not be in the same administrative group, an EtherChannel bundle would not form. Understanding this point is why it is important you understand the default administrative groups.

NOTE

You are not locked into accepting the default port allocations used for the default administrative groups. For example, you can manually configure a new administrative group that includes ports 2/4 and 2/5, which would enable an EtherChannel bundle to form in the scenario just described.

The main point to take from the discussion about default administrative groups is that you are strongly recommended to configure your own administrative groups, which are specific to your requirements, encompassing the required amount of links and the desired ports that make up the bundle. Configuring your own administrative groups ensures your EtherChannel bundles form as intended, without any nasty surprises.

To create a new EtherChannel bundle on CatOS, use the set port channel command.

 Console> (enable) set port channel port-list {auto | desirable | on | off}   [silent | non-silent] 

The port-list parameter defines the list of ports that you wish to assign to the bundle. You must then specify the appropriate PAgP mode and optionally may specify silent or non-silent operation for desirable or auto modes (you can't configure silent or non-silent operation for a PAgP mode of on or off). After entering in the command, a new EtherChannel bundle is created and automatically assigned an administrative group ID.

Example 3-28 demonstrates configuring an EtherChannel bundle that includes ports 2/1 and 2/2 on Switch-B in Figure 3-16.

Example 3-115. Configuring an EtherChannel Bundle on CatOS

 Switch-B> (enable) set port channel 2/1-2 auto Port(s) 2/1-2 are assigned to admin group 14. Port(s) 2/1-2 channel mode set to auto. 

In Example 3-28, notice that the new EtherChannel bundle is automatically assigned an administrative group ID of 14. This ID is one higher than the highest administrative group ID shown in Example 3-27 (CatOS automatically increments the administrative group ID created to ensure each administrative group ID is unique).

On CatOS, it is important that you do not confuse the following command with the command issued in Example 3-28:

 Console> (enable) set port channel port-list mode {auto | desirable | on | off}   [silent | non-silent] 

Notice that this command includes the mode keyword. When the mode keyword is specified, no EtherChannel bundle is actually configured (i.e., no administrative group is created). Only the PAgP mode of the ports listed is modified according to the mode specified. In other words, using the mode keyword enables you to modify the PAgP mode of ports in existing administrative groups (the port-list specified can span multiple administrative groups) instead of creating a new administrative group. Example 3-29 demonstrates modifying the EtherChannel bundle created in Example 3-28 (administrative group 14) to use a PAgP mode of desirable.

Example 3-116. Modifying PAgP Mode on CatOS

 Switch-B> (enable) set port channel 2/1-2 mode desirable Port(s) 2/1-2 channel mode set to desirable. 

If you compare Example 3-28 and Example 3-29, notice that in Example 3-29, a new administrative group is not created. Instead, only the PAgP mode of the ports has been modified.

Another variant of the set port channel command has the following syntax:

 Console> (enable) set port channel port-list [admin-group] 

This command allows you to create a new EtherChannel and manually specify the administrative group ID using the admin-group parameter. This command may be useful if you configure many EtherChannel bundles on your switch and wish to use a numbering scheme to ease management of administrative groups.

NOTE

Valid values for the administrative group ID include 1 to 1024.

Example 3-30 demonstrates using this command on Switch-B in Figure 3-16 to create an EtherChannel bundle that has an administrative group ID of 100.

Example 3-117. Manually Specifying Administrative Group ID on CatOS

 Switch-B> (enable) set port channel 2/1-2 100 

Port(s) 2/1-2 are assigned to administrative group 100.

To verify that you have created an EtherChannel bundle, use the show port channel command, as demonstrated in Example 3-31 on Switch-B.

Example 3-118. Verifying EtherChannel Configuration

 Switch-B> (console) show port channel Port  Status     Channel              Admin Ch                  Mode                 Group Id ----- ---------- -------------------- ----- -----  2/1  connected  desirable silent       100   801  2/2  connected  desirable silent       100   801 Port  Device-ID                       Port-ID                   Platform ----- ------------------------------- ------------------------- ----------------  2/1  Switch-A                        Fa0/1                     cisco WS-C3550-24  2/2  Switch-A                        Fa0/2                     cisco WS-C3550-24 

The first section of Example 3-31 indicates the current status of the EtherChannel bundle. You can see that both ports have a status of "connected," a channel mode of "desirable silent," and an administrative group ID of 100. The second section of Example 3-31 indicates port-specific information within the bundle. You can see that Switch-A (as indicated by the Device-ID column) is connected to both ports 2/1 and 2/2, that port 2/1 is connected to interface fa0/1, and that port 2/2 is connected to interface fa0/2 on Switch-A (as indicated by the Port-ID column). You can even see the Cisco Catalyst model of switch connected to Switch-B. Most of this information is exchanged via the PAgP negotiation process.

NOTE

Platform information is derived from CDP information advertised via Cisco devices, not from PAgP.

Configuring EtherChannel Load Distribution

EtherChannel load distribution is an important consideration because it affects how EtherChannel bundles perform in the network. How load distribution is configured is very much dependant on the network topology and the load distribution methods supported on the switches used in the network topology (see Table 3-6 for more information).

Cisco IOS Configuration

To configure the EtherChannel load sharing mechanism on Cisco IOS, use the port-channel load-balance global configuration mode command, as shown:

 Switch(config)# port-channel load-balance {src-mac | dst-mac | src-dst-mac |   src-ip | dst-ip | src-dst-ip | src-port | dst-port | src-dst-port} 

Notice that a wide variety of load distribution mechanisms are indicated; however, depending on the switch platform you are using, the available load distribution methods vary as follows:

• Cisco Catalyst 2900XL/3500XL and Catalyst 2950/3550 switches support load distribution based only upon Layer 2 MAC addresses.

• Native IOS Catalyst 6000/6500 switches that include a PFC1 support load distribution based upon Layer 2 MAC addresses and Layer 3 IP addresses.

• Cisco Catalyst 4000 Supervisor 3/4 and native IOS Catalyst 6000/6500 switches that include a PFC2 support load distribution based upon Layer 2 MAC addresses, Layer 3 IP addresses, and Layer 4 TCP/UDP ports.

In this scenario, Switch-A is a Catalyst 3550 switch; hence, only load distribution based upon Layer 2 MAC addresses can be configured. This restriction means that on Switch-A the port-channel load-balance global configuration mode command has the following syntax:

 Switch(config)# port-channel load-balance {src-mac | dst-mac} 

The default is to load share based upon source MAC address. Example 3-32 shows how to configure load sharing on Switch-A based upon the destination MAC address of each frame sent across any EtherChannel bundle (this setting applies globally for all EtherChannel bundles).

Example 3-119. Configuring EtherChannel Load Distribution on Cisco IOS

 Switch-A# configure terminal Switch-A(config)# port-channel load-balance dst-mac 

WARNING

There is one big caveat to configuring load sharing on the Cisco Catalyst series switches, whether they are CatOS-based or Cisco IOS-basedthe load sharing method is implemented globally for all bundles. This restriction means you must choose the most optimal method for all scenarios.

CatOS Configuration

To configure the EtherChannel load sharing mechanism on CatOS, use the set port channel all distribution command as shown:

 Console> (enable) set port channel all distribution {ip | mac | session}   [source | destination | both] 

The first set of configurable parameters specifies whether Layer 2 (indicated by the mac keyword), Layer 3 (indicated by the ip keyword), or Layer 4 (indicated by the session keyword) addressing should be used. The final set of configurable parameters indicates whether the source (indicated by the source keyword), destination (indicated by the destination keyword), or both source and destination (indicated by the both keyword) addressing should be used. Notice that a wide variety of load distribution mechanisms are indicated; however, depending on the switch platform you are using, the available load distribution methods vary as follows:

• Cisco Catalyst 2900/4000 and Catalyst 5000/5500 switches support only a single load distribution method based upon Layer 2 MAC addresses that cannot be modified.

• Catalyst 6000/6500 switches that include a PFC1 support load distribution based upon Layer 2 MAC addresses and Layer 3 IP addresses.

• Catalyst 6000/6500 switches that include a PFC2 support load distribution based upon Layer 2 MAC addresses, Layer 3 IP addresses, and Layer 4 TCP/UDP ports

In this scenario, Switch-A is a Catalyst 6500 switch with a Supervisor 2/PFC2; hence, all load distribution mechanisms are supported. The default load distribution mechanism on this switch is based upon both source IP address and destination IP address (i.e., set port channel all distribution ip both). Assume in Figure 3-16 that many client PCs are connected to Switch-B and that these PCs make client/server connections to servers attached to Switch-A. This situation means that most frames sent from Switch-B to Switch-A include a random Layer 4 source port (because client ports are normally chosen randomly) and a well-known Layer 4 destination port (because server ports normally listen on fixed, well-known ports). Configuring load sharing based upon Layer 4 source port should ensure an even load distribution because each source port value is essentially a random value. Example 3-33 shows how to configure load sharing on Switch-B based upon only the source Layer 4 ports of frames sent across any EtherChannel bundle (this setting applies globally for all EtherChannel bundles).

Example 3-120. Configuring EtherChannel Load Distribution on Cisco IOS

 Switch-B> (config) set port channel all distribution session source Channel distribution is set to mac both. 

Verifying EtherChannel Configuration

Once you have completed your EtherChannel configuration, verify each configured bundle has come up and that the correct ports are included in each bundle. The first step in verifying EtherChannel configuration is to verify the bundle as a whole has been created and is up. On Cisco IOS, you can use the show etherchannel summary command (see Example 3-25) to get a quick view of the status of each EtherChannel bundle, and on CatOS, you can use the show port channel command (see Example 3-31) to achieve a similar result.

Once you have checked the overall status of EtherChannel bundles, if you discover any problems, you can use the show etherchannel port command on Cisco IOS to view port-specific information about ports in an EtherChannel bundle, as demonstrated in Example 3-34.

Example 3-121. Verifying a Port Within an EtherChannel Bundle on Switch-A

 Switch-A# show etherchannel port                 Channel-group listing:                 ----------------------- Group: 1 ----------                 Ports in the group:                 ------------------- Port: Fa0/1 ------------ Port state    = Up Mstr In-Bndl Channel group = 1           Mode = Desirable-Sl     Gcchange = 0 Port-channel  = Po1         GC   = 0x00010001    Pseudo port-channel = Po1 Port index    = 0           Load = 0x00 Flags:  S - Device is sending Slow hello.  C - Device is in Consistent state.         A - Device is in Auto mode.        P - Device learns on physical port.         d - PAgP is down. Timers: H - Hello timer is running.        Q - Quit timer is running.         S - Switching timer is running.    I - Interface timer is running. Local information:                                 Hello    Partner  PAgP     Learning  Group Port      Flags State   Timers  Interval Count   Priority   Method  Ifindex Fa0/1     SC    U6/S7   H       30s      1        128        Any      29 Partner's information:           Partner              Partner          Partner         Partner Group Port      Name                 Device ID        Port       Age  Flags   Cap. Fa0/1     JAB03350EJR(Switch-B 0030.2448.d41b   2/1         23s SC      2 Age of the port in the current state: 00d:00h:27m:15s ... <Output Truncated> ... 

In Example 3-34, you can see that a lot of information is provided. Each EtherChannel bundle is listed, followed by specific information for each port within the bundle. Within the information listed for each port, local information is provided, along with information about the remote device connected (see the "Partner's information" section).

At this point, you have verified that the EtherChannel bundle has formed, which means that the bundle should be up as a trunk. Example 3-35 demonstrates using the show interface trunk command to verify that the bundle has come up as an 802.1Q trunk between Switch-A and Switch-B.

Example 3-122. Verifying the Trunk Between Switch-A and Switch-B Has Come Up on the EtherChannel Bundle

 Switch-A# show interface trunk Port      Mode         Encapsulation  Status        Native vlan Po1       desirable    802.1q         trunking      10 Port      Vlans allowed on trunk Po1       2-5,10 Port      Vlans allowed and active in management domain Po1       2-5,10 Port      Vlans in spanning tree forwarding state and not pruned Po1       2,3,5,10 

In Example 3-35, notice that the trunk port is now the port-channel 1 interface, indicating this interface is a logical Layer 2 interface.

As a final test, verify IP connectivity between the management interfaces on each switch and between devices within each VLAN.

Verifying Load Sharing and Redundancy

If you wish to verify that the load sharing and redundancy features are performing as you expect, you can use the show interfaces counters command on Cisco IOS to test load sharing and redundancy. Example 3-36 shows an example of the use of the show interfaces counters command.

Example 3-123. Verifying Load Balancing and Redundancy on a Cisco IOS Switch

 Switch-A# show interfaces counters Port            InOctets   InUcastPkts   InMcastPkts   InBcastPkts Fa0/1             430066           443          5387             3 Fa0/2              85234           195           765             0 ...<Output Truncated> ... Port           OutOctets  OutUcastPkts  OutMcastPkts  OutBcastPkts Fa0/1             181410           281          1189            10 Fa0/2             105111           502           440            69 ...<Output Truncated> ... 

Example 3-36 shows that you can determine load sharing based upon the unicast, multicast, and broadcast frame counts shown.

On CatOS you can use the show channel traffic command on CatOS to display traffic statistics for each physical interface within a bundle. This command displays both received and transmitted traffic, so you can verify how traffic is being load balanced locally over the channel (by viewing the transmitted statistics) and how it is being load balanced remotely over the channel (by viewing the received statistics). Example 3-37 shows a sample output of the show channel traffic command.

Example 3-124. Verifying Load Balancing on a CatOS Switch

 Switch-B> (enable) show channel traffic ChanId Port  Rx-Ucst Tx-Ucst Rx-Mcst Tx-Mcst Rx-Bcst Tx-Bcst ------ ----- ------- ------- ------- ------- ------- -------    100  2/1   62.50%  44.50%  50.00%  75.75%  46.00%   42.50%    100  2/2   37.50%  55.50%  50.00%  24.25%  54.00%   57.50% ... ... 

In Example 3-37, you can easily verify traffic load distribution. For example, 44.50 percent of sent unicast traffic is being sent over port 2/1 and is then sent over the bundle.