FC Flow Control and QoS

This section summarizes the flow-control and QoS mechanisms supported by FC.

FC Flow Control

The primary flow-control mechanism used in modern FC-SANs (Class 3 fabrics) is the Buffer-to-Buffer_Credit (BB_Credit) mechanism. The BB_Credit mechanism provides link-level flow control. The FLOGI procedure informs the peer port of the number of BB_Credits each N_Port and F_Port has available for frame reception. Likewise, the Exchange Link Parameters (ELP) procedure informs the peer port of the number of BB_Credits each E_Port has available for frame reception. Each time a port transmits a frame, the port decrements the BB_Credit counter associated with the peer port. If the BB_Credit counter reaches zero, no more frames may be transmitted until a Receiver_Ready (R_RDY) primitive signal is received. Each time an R_RDY is received, the receiving port increments the BB_Credit counter associated with the peer port. Each time a port processes a received frame, the port transmits an R_RDY to the peer port. The explicit, proactive nature of the BB_Credit mechanism ensures that no frames are ever dropped in FC-SANs because of link-level buffer overrun. However, line-rate throughput can be very difficult to achieve over long distances because of the high BB_Credit count requirement. Some of the line cards available for FC switches produced by Cisco Systems support thousands of BB_Credits on each port, thus enabling long-distance SAN interconnectivity over optical networks without compromising throughput. When FC-SANs are connected over long-distance optical networks, R_RDY signals are sometimes lost. When this occurs, throughput drops slowly over a long period. This phenomenon can be conceptualized as temporal droop. This phenomenon also can occur on native FC inter-switch links (ISLs), but the probability of occurrence is much lower with local connectivity. The FC-FS-2 specification defines a procedure called BB_Credit Recovery for detecting and recovering from temporal droop. For more information about FC flow control, readers are encouraged to consult the ANSI T11 FC-FS-2 and FC-BB-3 specifications.

FC switches produced by Cisco Systems also support a proprietary flow control feature called FC Congestion Control (FCC). Conceptually, FCC mimics the behavior of ICMP Source-Quench. When a port becomes congested, FCC signals the switch to which the source node is connected. The source switch then artificially slows the rate at which BB_Credits are transmitted to the source N_Port. Cisco Systems might submit FCC to ANSI for inclusion in a future FC standard.

FC QoS

FC supports several QoS mechanisms via fields in the FC header. The DSCP subfield in the CS_CTL/Priority field can be used to implement differentiated services similar to the IP DiffServ model. However, the FC-FS-2 specification currently reserves all values other than zero, which is assigned to best-effort service. The Preference subfield in the CS_CTL/Priority field can be used to implement a simple two-level priority system. The FC-FS-2 specification requires all Class 3 devices to support the Preference subfield. No requirement exists for every frame within a sequence or Exchange to have the same preference value. So, it is theoretically possible for frames to be delivered out of order based on inconsistent values in the Preference fields of frames within a sequence or Exchange. However, this scenario is not likely to occur because all FC Host Bus Adapter (HBA) vendors recognize the danger in such behavior. The Priority subfield in the CS_CTL/Priority field can be used to implement a multi-level priority system. Again, no requirement exists for every frame within a sequence or Exchange to have the same priority value, so out-of-order frame delivery is theoretically possible (though improbable). The Preemption subfield in the CS_CTL/Priority field can be used to preempt a Class 1 or Class 6 connection to allow Class 3 frames to be forwarded. No modern FC switches support Class 1 or Class 6 traffic, so the Preemption field is never used. For more information about FC QoS, readers are encouraged to consult the ANSI T11 FC-FS-2 specification.