FCP Operational Details

This section provides an in-depth exploration of the operational details of FCP. This section complements and builds upon the FCP overview provided in Chapter 4. You are also encouraged to consult the ANSI T11 FC-DA specification series to fully understand which combinations of FC and FCP features are permitted in real-world deployments.

FCP Addressing Scheme

FCP leverages the FC addressing scheme described in Chapter 5. No additional names or addresses are defined by the ANSI T10 FCP-3 specification.

FCP Name Assignment and Resolution

Name assignment for FCP devices and ports occurs as described in Chapter 5. For name resolution, FCP leverages the FCNS as described in Chapters 3 and 5. The FCP-3 specification suggests using a service-oriented approach during initial discovery of fabric attached devices. However, discovery behavior is not mandated by the FCP-3 specification. So, FCP devices may query the FCNS in a wide variety of ways. FCP devices may also leverage the ELS commands described in Chapter 5.

FCP Address Assignment and Resolution

Address assignment for FCP ports occurs as described in Chapter 5. No additional procedures or special requirements are defined by the FCP-3 specification.

FCP Session Establishment

Unlike iSCSI, FCP does not implement multiple session types, phases and stages. Instead of establishing a discovery session with the target device, FCP initiators establish a session with the FCNS to perform discovery operations. Likewise, targets establish a session with the FCNS. This session is long-lived and remains active until the initiator or target goes offline. Initiators establish the same type of FCP session with target devices to perform I/O operations.

An FCP session is established with a simple two-way handshake. FCP processes establish communication via the process login (PRLI) ELS command defined in the FC-LS specification. Only initiators may send a PRLI Request. A single PRLI Request followed by a single LS_ACC ELS can establish a session between two FCP devices. Alternately, a single PRLI Request and a single LS_ACC ELS may be exchanged for the purpose of discovering the FCP operating parameters (called service parameters in FCP parlance) supported by a pair of FCP devices. In this case, another PRLI Request and another LS_ACC ELS are required to establish a session. Like iSCSI, some FCP service parameters are negotiated, and others are simply declared. Negotiated FCP parameters are called requirements, and all others are called capabilities. After a session is established, the relationship between the communicating FCP processes is called an image pair. An FCP image pair must exist before SCSI data transfer is permitted between FC devices. As with PLOGI, PRLI may be performed explicitly or implicitly. Only explicit PRLI is described in this book for the sake of clarity.

Whereas iSCSI natively supports the exchange of all required operating parameters to establish a new session, FCP relies on SCSI for the exchange of certain service parameters. This is accomplished via the scsi mode sense and mode select commands. The mode sense command is used to discover the service parameters supported by a target device or logical units within a target device. The mode select command is used to inform a target device or logical units within a target device which service parameters to use. Because SCSI commands cannot be issued until after PRLI completes, the use of scsi mode commands to establish FCP service parameters requires modification of FCP session behavior after session establishment. The FCP-3 specification does not define how this modification should be accomplished. So, one of the logical unit device servers (usually LUN 0) must communicate scsi mode parameters to the FCP port(s) via proprietary means. This is handled within each SCSI device (no network communication). For more information about FCP's use of the scsi mode commands, see the FCP Login Parameters section of this chapter. For more information about FCP session establishment, readers are encouraged to consult the ANSI T10 FCP-3 and SPC-3 specifications, and the ANSI T11 FC-FS-2 and FC-LS specifications.

FCP Data Transfer Optimizations

FCP does not support equivalent functionality to iSCSI phase-collapse for the final data-in information unit (IU) and SCSI status. However, FCP supports equivalent functionality to iSCSI phase-collapse for first burst data. Support for unsolicited first burst data is negotiated via PRLI, but PRLI does not support negotiation of the first burst buffer size. Therefore, the first burst buffer size must be negotiated using the SCSI MODE commands with the Disconnect-Reconnect mode page or via proprietary means. FCP does not support immediate data. For more information about FCP first burst optimization, readers are encouraged to consult the ANSI T10 FCP-3 and SPC-3 specifications.

When DWDM or SONET is used to extend an FC-SAN, end-to-end latency can be sufficiently significant to affect performance. As a result, FC switch vendors and SAN extension vendors are responding with proprietary features that reduce round-trip delay in such environments. For example, FC switches produced by Cisco Systems can reduce round-trip delay for write data transfers via a feature called FC write acceleration (FCWA). FCWA provides local spoofing of the target signal (an FCP_XFER_RDY IU) that indicates readiness to receive write data. The switch at the remote site performs the necessary buffering to avoid dropped frames. Unlike the standardized FCP first burst optimization, FCWA operates on every data IU transmitted by an initiator. Another key difference in the behavior of these two optimizations is that FCWA does not eliminate the need for flow-control signaling, whereas unsolicited first burst data is sent without receiving a transfer request from the target.

FCP IU Formats

In FCP parlance, a protocol data unit is called an information unit. The FCP-3 specification defines five types of IU: FCP_CMND, FCP_DATA, FCP_XFER_RDY, FCP_RSP, and FCP_CONF. This section describes all five IUs in detail.

This section also describes the details of the link services most commonly used by FCP. As discussed in Chapter 5, the FC specifications define many link services that may be used by end nodes to interact with the FC-SAN and to manage communication with other end nodes. Three types of link service are defined: basic, extended, and FC-4. Each basic link service (BLS) command is composed of a single frame that is transmitted as part of an existing Exchange. Despite this, BLS commands are ignored with regard to the ability to mix information categories within a single sequence as negotiated during PLOGI. The response to a BLS command is also a single frame transmitted as part of an existing Exchange. BLSs are defined in the FC-FS specification series. The BLS most commonly used by FCP is abort sequence (ABTS). As stated in Chapter 5, an ELS may be composed of one or more frames per direction transmitted as a single sequence per direction within a new Exchange. Most ELSs are defined in the FC-LS specification. The ELSs most commonly used by FCP include PRLI and read exchange concise (REC). An FC-4 link service may be composed of one or more frames and must be transmitted as a new Exchange. The framework for all FC-4 link services is defined in the FC-LS specification, but the specific functionality of each FC-4 link service is defined in an FC-4 protocol specification. The FCP-3 specification defines only one FC-4 link service called sequence retransmission request (SRR). This section describes the ABTS, PRLI, REC, and SRR link services in detail.

FCP IUs are encapsulated within the Data field of the FC frame. An FCP IU that exceeds the maximum size of the Data field is sent as a multi-frame Sequence. Each FCP IU is transmitted as a single Sequence. Additionally, each Sequence composes a single FCP IU. This one-to-one mapping contrasts the iSCSI model. Fields within the FC Header indicate the type of FCP IU contained in the Data field. Table 8-10 summarizes the values of the relevant FC Header fields.

Table 8-10. FC Header Field Values for FCP IUs

FCP IU	R_CTL Routing	R_CTL Information Category	Type	F_CTL Relative Offset Present	DF_CTL
FCP_CMND	0000b	0110b	0x08	0b	0x00 or 0x40
FCP_DATA	0000b	0001b	0x08	1b	0x00 or 0x40
FCP_XFER_RDY	0000b	0101b	0x08	0b	0x00 or 0x40
FCP_RSP	0000b	0111b	0x08	0b	0x00 or 0x40
FCP_CONF	0000b	0011b	0x08	0b	0x00 or 0x40

Only one of the members of an FCP image pair may transmit FCP IUs at any point in time. The sequence initiative (SI) bit in the F_CTL field in the FC Header controls which FCP device may transmit. To transmit, an FCP device must hold the sequence initiative. If an FCP device has more than one FCP IU to transmit, it may choose to hold the sequence initiative after transmitting the last frame of a Sequence. Doing so allows the FCP device to transmit another FCP IU. This is known as Sequence streaming. When the FCP device has no more FCP IUs to transmit, it transfers the sequence initiative to the other FCP device. During bidirectional commands, the sequence initiative may be transferred many times at intervals determined by the participating FCP devices.

When an FC frame encapsulates an FCP_CMND IU, the Parameter field in the FC Header can contain a task identifier to assist command retry. If command retry is not supported, the Parameter field is set to 0. Command retry is discussed in the FCP Delivery Mechanisms section of this chapter. Unlike iSCSI, FCP uses a single command IU (FCP_CMND) for SCSI commands and TMF requests. Figure 8-16 illustrates the format of the FCP_CMND IU. Note that FCP IUs are word-oriented like FC frames, but the FCP specification series illustrates FCP IU formats using a byte-oriented format. This book also illustrates FCP IU formats using a byte-oriented format to maintain consistency with the FCP specification series.

Figure 8-16. FCP_CMND IU Format

A brief description of each field follows:

• FCP_LUN This is 64 bits (8 bytes) long. It contains the destination LUN. The format of this field complies with the SAM LUN format.

• Command Reference Number (CRN) This is 8 bits long. If in-order command delivery is enabled, the initiator assigns a unique reference number to each SCSI command issued within a session. Note that the FCP_CMND IU does not have a field to identify the SCSI task. Instead, each SCSI task is identified by the fully qualified exchange identifier (FQXID). The FQXID is composed of the S_ID, D_ID, OX_ID, and RX_ID fields in the FC Header. Thus, each FC Exchange represents a SCSI task. This requires SCSI linked commands to be transmitted in a single Exchange. A retransmitted FCP_CMND IU carries the same CRN as the original IU. However, a retransmitted FCP_CMND IU uses a new FQXID, so the initiator's internal mapping of the SCSI task tag to an FQXID must be updated. The FCP CRN is similar in function to the iSCSI CmdSN. The FC FQXID is similar in function to the iSCSI ITT.

• Reserved This is 1 bit.

• Priority This is 4 bits long. It determines the order of execution for tasks in a task manager's queue. This field is valid only for SIMPLE tasks.

• Task Attribute This is 3 bits long. A value of 0 indicates a SIMPLE task. A value of 1 indicates a HEAD OF QUEUE task. A value of 2 indicates an ORDERED task. A value of 4 indicates an ACA task. All other values are reserved. For more information about SCSI Task Attributes, see the ANSI T10 SAM-3 specification.

• Task Management Flags This is 8 bits long. This field is used to request a TMF. If any bit in this field is set to 1, a TMF is requested. When a TMF is requested, the FCP_CMND IU does not encapsulate a SCSI command. Thus, the Task Attribute, Additional FCP_CDB Length, RDDATA, WRDATA, FCP_CDB, Additional FCP_CDB, FCP_DL, and FCP_Bidirectional_Read_DL fields are not used. No more than one bit in the Task Management Flags field may be set to 1 in a given FCP_CMND IU. Bit 1 represents the Abort Task Set TMF. Bit 2 represents the Clear Task Set TMF. Bit 4 represents the Logical Unit Reset TMF. Bit 6 represents the Clear ACA TMF. All other bits are reserved. Note that the Abort Task TMF is not supported via this field. Instead, the ABTS BLS is used. Despite its name, the ABTS BLS can be used to abort a single sequence or an entire Exchange. When the FCP_CMND IU encapsulates a SCSI command, the Task Management Flags field must be set to 0.

• Additional FCP_CDB Length This is 6 bits long. This field indicates the length of the Additional FCP_CDB field expressed in 4-byte words. When the CDB length is 16 bytes or less, this field is set to 0. When a TMF is requested, this field is set to 0.

• RDDATA This is 1 bit. It indicates a read command when set to 1. For bidirectional commands, both the RDDATA and WRDATA bits are set to 1.

• WRDATA This is 1 bit. It indicates a write command when set to 1. For bidirectional commands, both the RDDATA and WRDATA bits are set to 1.

• FCP_CDB This is 128 bits (16 bytes) long. It contains a SCSI CDB if the Task Management Flags field is set to 0. When a CDB shorter than 16 bytes is sent, this field is padded. The value of padding is not defined by the FCP-3 specification. Presumably, zeros should be used for padding. When a CDB longer than 16 bytes is sent, this field contains the first 16 bytes of the CDB.

• Additional FCP_CDB This is variable in length. When a CDB longer than 16 bytes is sent, this field contains all bytes of the CDB except the first 16 bytes. All CDBs longer than 16 bytes must end on a 4-byte word boundary, so this field does not require padding. When the CDB length is 16 bytes or less, this field is omitted from the FCP_CMND IU. Likewise, when a TMF is requested, this field is omitted from the FCP_CMND IU.

• FCP_DL This is 32 bits long. It indicates the total amount of data expected to be transferred unidirectionally by this command. This field is expressed in bytes. When the data transfer is bidirectional, this field represents the write data, and the FCP_ Bidirectional_Read_DL field must be present in the FCP_CMND IU. This field is set to 0 for certain commands that do not transfer data. This field represents an estimate. After all data is transferred, the target informs the initiator of how much data was transferred.

• FCP_BIDIRECTIONAL_READ_DL This is 32 bits long. It indicates the total amount of read data expected to be transferred by a bidirectional command. When the SCSI command is unidirectional, this field is omitted from the FCP_CMND IU. This field represents an estimate. After all data is transferred, the target informs the initiator how much data was transferred.

Unlike iSCSI, FCP uses a single IU (FCP_DATA) for both data-out and data-in operations. The only difference in IU format for data-out versus data-in operations is the manner in which the SI bit in the FC Header is handled. For data-out operations, the sequence initiative is transferred from initiator to target after transmission of each FCP_DATA IU. This enables the target to transmit an FCP_XFER_RDY IU to continue the operation or an FCP_RSP IU to complete the operation. For data-in operations, the sequence initiative is held by the target after transmission of each FCP_DATA IU. This enables the target to transmit another FCP_DATA IU to continue the operation or an FCP_RSP IU to complete the operation. For all FCP_DATA IUs, the Parameter field in the FC Header contains a relative offset value. The FCP_DATA IU does not have a defined format within the Data field of the FC frame. The receiver uses only the FC Header to identify an FCP_DATA IU, and ULP data is directly encapsulated in the Data field of the FC frame. An FCP_DATA IU may not be sent with a payload of 0 bytes.

Note

The FC-FS-2 specification mandates the Information Category sub-field of the R_CTL field in the FC Header must be set to solicited data even when the initiator sends unsolicited first burst data.

When an FC frame encapsulates an FCP_XFER_RDY IU, the Parameter field in the FC Header is set to 0. In contrast to the iSCSI model, FCP implements a one-to-one relationship between FCP_XFER_RDY IUs and FCP_DATA IUs. The FC-FS-2 specification categorizes the FCP_XFER_RDY IU as a Data Descriptor. The FC-FS-2 specification also defines the general format that all Data Descriptors must use. Figure 8-17 illustrates the general format of Data Descriptors. Figure 8-18 illustrates the format of the FCP_XFER_RDY IU.

Figure 8-17. Data Descriptor Format

Figure 8-18. FCP_XFER_RDY IU Format

A brief description of each field follows:

• FCP_DATA_RO This is 32 bits long. This field indicates the position of the first byte of data requested by this IU relative to the first byte of all the data transferred by the scsi command.

• FCP_BURST_LEN This is 32 bits long. This field indicates how much data should be transferred in response to this FCP_XFER_RDY IU. This field is expressed in bytes. The value of this field cannot be 0 and cannot exceed the negotiated value of maximum burst size (see the FCP Login Parameters section of this chapter).

• Reserved This is 32 bits long.

The final result of each SCSI command is a SCSI status indicator delivered in an FCP_RSP IU. The FCP_RSP IU also conveys FCP status for protocol operations. When an FC frame encapsulates an FCP_RSP IU, the Parameter field in the FC Header is set to 0. Unlike iSCSI, FCP uses a single response IU (FCP_RSP) for SCSI commands and TMF requests. Figure 8-19 illustrates the format of the FCP_RSP IU.

Figure 8-19. FCP_RSP IU Format

A brief description of each field follows:

• Reserved This is 64 bits (8 bytes) long.

• Retry Delay Timer This is 16 bits long. This field contains one of the retry delay timer codes defined in the SAM-4 specification. These codes provide additional information to the initiator regarding why a command failed and how long to wait before retrying the command.

• FCP_BIDI_RSP This is one bit. This field indicates whether the FCP_RSP IU provides status for a bidirectional command. When this bit is set to 1, the FCP_BIDI_ READ_RESID_UNDER, FCP_BIDI_READ_RESID_OVER, and FCP_ BIDIRECTIONAL_READ_RESID fields are valid. When this bit is set to 0, the FCP_BIDI_READ_RESID_UNDER, FCP_BIDI_READ_RESID_OVER, and FCP_BIDIRECTIONAL_READ_RESID fields are ignored.

• FCP_BIDI_READ_RESID_UNDER This is one bit. When this bit is set to 1, the FCP_BIDIRECTIONAL_READ_RESID field is valid. When this bit is set to 1, the FCP_BIDI_READ_RESID_OVER bit must be set to 0. When this bit is set to 0, the command transferred at least as many read bytes as expected.

• FCP_BIDI_READ_RESID_OVER This is 1 bit. When this bit is set to 1, the FCP_BIDIRECTIONAL_READ_RESID field is valid. When this bit is set to 1, the FCP_BIDI_READ_RESID_UNDER bit must be set to 0. When this bit is set to 0, the command did not transfer more read bytes than expected.

• FCP_CONF_REQ This is 1 bit. When this bit is set to 1, the target is requesting an FCP_CONF IU from the initiator. When this bit is set to 0, the initiator does not send an FCP_CONF IU to the target.

• FCP_RESID_UNDER This is 1 bit. When this bit is set to 1, the FCP_RESID field is valid. When this bit is set to 1, the FCP_RESID_OVER bit must be set to 0. When this bit is set to 0, the command transferred at least as many bytes as expected. For bidirectional commands, this bit represents the write direction.

• FCP_RESID_OVER This is 1 bit. When this bit is set to 1, the FCP_RESID field is valid. When this bit is set to 1, the FCP_RESID_UNDER bit must be set to 0. When this bit is set to 0, the command did not transfer more bytes than expected. For bidirectional commands, this bit represents the write direction.

• FCP_SNS_LEN_VALID This is 1 bit. When this bit is set to 1, the FCP_SNS_LEN and FCP_SNS_INFO fields are valid. When this bit is set to 0, the FCP_SNS_LEN and FCP_SNS_INFO fields are ignored.

• FCP_RSP_LEN_VALID This is 1 bit. For TMF requests, this bit must be set to 1 because TMF completion status is provided via the FCP_RSP_INFO field. So, FCP delivery status cannot be explicitly communicated via this field when also communicating TMF completion status. For SCSI commands, this bit may be set to 1 or 0. The remainder of this paragraph focuses on transport of SCSI commands. This bit indicates the presence or absence of FCP errors. In other words, this bit is to the SCSI service delivery subsystem what the SCSI status code is to SAL. When this bit is set to 1, the FCP_RSP_LEN and FCP_RSP_INFO fields are valid, and the SCSI Status Code field is ignored. This is roughly equivalent to a SCSI service response of SERVICE DELIVERY OR TARGET FAILURE. This is also roughly equivalent to an iSCSI response code of 0x01 (target failure). Setting this bit to 0 indicates that the target has completed processing the command from the FCP perspective. When this bit is set to 0, the FCP_RSP_LEN and FCP_RSP_INFO fields are ignored, and the SCSI Status Code field is valid. This is roughly equivalent to a SCSI service response of LINKED COMMAND COMPLETE or TASK COMPLETE. This is also roughly equivalent to an iSCSI response code of 0x00 (command completed at target). Setting this bit to 0 conveys FCP success but does not imply SCSI success.

• SCSI Status Code This is 8 bits long. This field contains a status code that provides more detail about the final status of the SCSI command and the state of the logical unit that executed the command. This field is valid only if the FCP_RSP_LEN_VALID bit is set to 0. Even if the FCP_RSP_LEN_VALID bit is set to 0, the target might not have successfully processed the command. If the status code indicates failure to successfully process the command, SCSI sense data is included in the FCP_SNS_INFO field. All FCP devices must support SCSI autosense. Like iSCSI, FCP uses the status codes defined by the SAM (see Table 8-3).

• FCP_RESID This is 32 bits long. When the FCP_RESID_UNDER bit is set to 1, this field indicates the number of bytes that were expected but not transferred. When the FCP_RESID_OVER bit is set to 1, this field indicates the number of bytes that were not transferred because they were not expected. Unexpected bytes cannot be transferred because the receiver does not have sufficient receive buffer space (as determined by the FCP_DL field in the FCP_CMND IU). When both the FCP_RESID_UNDER bit and the FCP_RESID_OVER bit are set to 0, this field is ignored.

• FCP_SNS_LEN This is 32 bits long. When the FCP_SNS_LEN_VALID bit is set to 1, this field indicates the length of the FCP_SNS_INFO field expressed in bytes. When the FCP_SNS_LEN_VALID bit is set to 0, this field is ignored.

• FCP_RSP_LEN This is 32 bits long. When the FCP_RSP_LEN_VALID bit is set to 1, this field indicates the length of the FCP_RSP_INFO field expressed in bytes. When the FCP_RSP_LEN_VALID bit is set to 0, this field is ignored. Valid values include 4 and 8. All other values are invalid.

• FCP_RSP_INFO This is variable in length. It currently may be either 4 or 8 bytes long. Figure 8-20 illustrates the 8-byte format of this field. When the FCP_RSP_LEN_VALID bit is set to 1, this field can contain an FCP response code that provides TMF completion status or SCSI service delivery subsystem diagnostic information about an FCP error (the FCP equivalent to SCSI autosense data). Table 8-11 summarizes the FCP response codes defined by the FCP-3 specification. All response codes excluded from Table 8-11 are reserved. When the FCP_RSP_LEN_VALID bit is set to 0, this field is omitted from the FCP_RSP IU.

  Figure 8-20. FCP_RSP_INFO Field Format

  

• FCP_SNS_INFO This is variable in length. When the FCP_SNS_LEN_VALID bit is set to 1, this field contains SCSI autosense data. When the FCP_SNS_ LEN_VALID bit is set to 0, this field is omitted from the FCP_RSP IU.

• FCP_BIDIRECTIONAL_READ_RESID This is 32 bits long. When the FCP_BIDI_READ_RESID_UNDER bit is set to 1, this field indicates the number of read bytes that were expected but not transferred. When the FCP_BIDI_READ_ RESID_OVER bit is set to 1, this field indicates the number of read bytes that were not transferred because they were not expected. Unexpected read bytes cannot be transferred because the initiator does not have sufficient receive buffer space (as determined by the FCP_BIDIRECTIONAL_READ_DL field in the FCP_CMND IU). When both the FCP_BIDI_READ_RESID_UNDER bit and the FCP_BIDI_READ_RESID_OVER bit are set to 0, this field is ignored.

Table 8-11. FCP Response Codes

Response Code	Description
0x00	TMF Complete
0x01	FCP_DATA Length Different Than FCP_BURST_LEN
0x02	FCP_CMND Fields Invalid
0x03	FCP_DATA Parameter Mismatch With FCP_DATA_RO
0x04	TMF Rejected
0x05	TMF Failed
0x09	TMF Incorrect LUN

The FCP_CONF IU is sent by an initiator only when requested via the FCP_CONF_REQ bit in the FCP_RSP IU. The FCP_CONF IU confirms the initiator received the referenced FCP_RSP IU. The target associates an FCP_CONF IU with the appropriate FCP_RSP IU via the FQXID. For all FCP_CONF IUs, the Parameter field in the FC Header is set to 0. The FCP_CONF IU does not have a defined format within the Data field of the FC frame. Additionally, the FCP_CONF IU has no payload. The target uses only the FC Header to determine a given FC frame is actually an FCP_CONF IU. The FCP_CONF IU is not supported for TMF requests. Similarly, the FCP_CONF IU is not supported for intermediate commands in a chain of SCSI-linked commands. For SCSI-linked commands, the FCP_CONF IU may be requested for only the last command of the task.

The PRLI ELS is the explicit mechanism by which a session is established between two FCP devices. Support for PRLI is optional. PRLI is the primary, but not the only, mechanism by which service parameters are exchanged. Only initiators can send a PRLI. Possible responses to PRLI include LS_ACC and Link Service Reject (LS_RJT). PRLI and its associated responses are each encapsulated in the Data field of an FC frame. The fields in the FC Header indicate the payload is a PRLI, LS_ACC, or LS_RJT. Table 8-12 summarizes the values of the relevant FC Header fields. A single format is defined in the FC-LS specification for both PRLI and its associated LS_ACC. Figure 8-21 illustrates the format of PRLI and its associated LS_ACC.

Figure 8-21. PRLI and Associated LS_ACC ELS Format

Table 8-12. FC Header Field Values for PRLI and LS_ACC ELS

ELS	R_CTL Routing	R_CTL Information Category	Type	F_CTL Relative Offset Present	DF_CTL
PRLI	0010b	0010b	0x01	0b	0x00 or 0x40
LS_ACC	0010b	0011b	0x01	0b	0x00 or 0x40
LS_RJT	0010b	0011b	0x01	0b	0x00 or 0x40

A brief description of each field follows:

• LS Command Code This is 8 bits long. For a PRLI Request, this field is set to 0x20. For an LS_ACC, this field is set to 0x02.

• Page Length This is 8 bits long. This field indicates the length of each Service Parameter Page expressed in bytes. This field is set to 0x10, so each Service Parameter Page is 16 bytes (4 words) long.

• Payload Length This is 16 bits long. This field indicates the total length of the PRLI Request or LS_ACC expressed in bytes. Valid values range from 20 to 65,532.

• Service Parameter Pages This is variable in length. This field may contain one or more Service Parameter Pages, but no more than one Service Parameter Page may be sent per image pair. The FC-LS specification defines the general format of the PRLI Service Parameter Page and the LS_ACC Service Parameter Page.

Like iSCSI, FCP negotiates some service parameters, while others are merely declared. The FCP-3 specification defines the specific format of the PRLI Service Parameter Page. Figure 8-22 illustrates the FCP-specific format of the PRLI Service Parameter Page.

Figure 8-22. PRLI Service Parameter Page Format for FCP

A brief description of each field follows:

• Type Code This is 8 bits long. This field is set to 0x08 and identifies the FC-4 protocol as FCP.

• Reserved This is 8 bits long.

• ORIGINATOR PROCESS_ASSOCIATOR VALID (OPAV) This is 1 bit. This bit is always set to 0, indicating that the ORIGINATOR PROCESS_ ASSOCIATOR field is not used.

• RESPONDER PROCESS_ASSOCIATOR VALID (RPAV) This is 1 bit. This bit is always set to 0, indicating that the RESPONDER PROCESS_ASSOCIATOR field is not used.

• ESTABLISH IMAGE PAIR (EIP) This is 1 bit. When this bit is set to 1, the initiator requests both the exchange of service parameters and the establishment of an image pair. When this bit is set to 0, the initiator requests only the exchange of service parameters.

• Reserved This is 13 bits long.

• ORIGINATOR PROCESS_ASSOCIATOR This is 32 bits long. It is not used.

• RESPONDER PROCESS_ASSOCIATOR This is 32 bits long. It is not used.

• Reserved This is 22 bits long.

• Service Parameters This is 10 bits long. It contains nine sub-fields.

• TASK RETRY IDENTIFICATION REQUESTED This is 1 bit. When this bit is set to 1, the initiator requests support for task retry identification. If the target agrees, the Parameter field in the FC Header of each FCP_CMND IU contains a task retry identifier. When this bit is set to 0, the initiator does not support task retry identification, and the Parameter field in the FC Header of each FCP_CMND IU is set to 0.

• RETRY This is 1 bit. When this bit is set to 1, the initiator requests support for retransmission of sequences that experience errors. If the target agrees, the SRR link service is used. When this bit is set to 0, the initiator does not support retransmission of sequences, and the SRR link service is not used.

• CONFIRMED COMPLETION ALLOWED This is 1 bit. When this bit is set to 1, the initiator declares support for the FCP_CONF IU, and the target may request confirmation via the FCP_CONF_REQ bit. When this bit is set to 0, the initiator does not support the FCP_CONF IU, and the target may not request confirmation via the FCP_CONF_REQ bit.

• DATA OVERLAY ALLOWED This is 1 bit. When this bit is set to 1, the initiator declares support for data overlay. Data overlay is the transfer of data to or from a single buffer offset multiple times per SCSI command. When this bit is set to 0, the initiator does not support data overlay, and the target must transfer data to or from a sequentially increasing buffer offset.

• INITIATOR FUNCTION This is 1 bit. When this bit is set to 1, initiator functionality is supported. When this bit is set to 0, initiator functionality is not supported. Because the PRLI Request ELS can be sent only by initiators, this bit is always set to 1.

• TARGET FUNCTION This is 1 bit. When this bit is set to 1, target functionality is supported. When this bit is set to 0, target functionality is not supported. Some devices, such as storage array ports used for replication, support both initiator and target functionality simultaneously. Those devices set this bit to 1. Most hosts support only initiator functionality. They set this bit to 0.

• OBSOLETE This is 2 bits long. It is not used.

• READ FCP_XFER_RDY DISABLED This is 1 bit. This bit is always set to 1.

• WRITE FCP_XFER_RDY DISABLED This is 1 bit. Despite its misleading name, this bit is applicable only to first burst data. When this bit is set to 1, the initiator requests support for unsolicited data. If the target agrees, the initiator may send one unsolicited FCP_DATA IU per SCSI write command. When this bit is set to 0, the initiator does not support unsolicited data.

In the absence of errors, the target responds to PRLI with LS_ACC. The FCP-3 specification defines the specific format of the LS_ACC Service Parameter Page. Figure 8-23 illustrates the FCP-specific format of the LS_ACC Service Parameter Page.

Figure 8-23. LS_ACC Service Parameter Page Format for FCP

A brief description of each field follows:

• Type Code This is 8 bits long. This field is set to 0x08. It identifies the FC-4 protocol as FCP.

• Reserved This is 8 bits long.

• ORIGINATOR PROCESS_ASSOCIATOR VALID (OPAV) This is 1 bit. This bit is always set to 0, indicating that the ORIGINATOR PROCESS_ASSOCIATOR field is not used.

• RESPONDER PROCESS_ASSOCIATOR VALID (RPAV) This is 1 bit. This bit is always set to 0, indicating that the RESPONDER PROCESS_ASSOCIATOR field is not used.

• IMAGE PAIR ESTABLISHED (IPE) This is 1 bit. This bit is valid only if the EIP bit is set to 1 in the PRLI Request. When this bit is set to 1, the target confirms the establishment of an image pair. When this bit is set to 0, the target is only exchanging service parameters.

• Reserved This is 1 bit.

• Accept Response Code (ARC) This is 4 bits long. This field contains a code that confirms that the image pair is established, or provides diagnostic information when the image pair is not established. Table 8-13 summarizes the PRLI Accept Response Codes defined in the FC-LS specification. All response codes excluded from Table 8-13 are reserved.

  Table 8-13. PRLI Accept Response Codes

  Response Code	Description
  0001b	The PRLI Request was executed without error.
  0010b	The target has no resources available. The PRLI Request may be retried.
  0011b	The target is still initializing. The PRLI Request may be retried.
  0100b	This code does not apply to FCP.
  0101b	The target has been preconfigured such that it cannot establish the requested image pair. The PRLI Request may not be retried.
  0110b	The PRLI Request was executed, but some service parameters were not set as requested.
  0111b	The target cannot process a multi-page PRLI Request. The PRLI Request may be retried as multiple single-page PRLI Requests.
  1000b	One or more service parameters are invalid.

  
  

• Reserved This is 8 bits long.

• ORIGINATOR PROCESS_ASSOCIATOR This is 32 bits long. It is not used.

• RESPONDER PROCESS_ASSOCIATOR This is 32 bits long. It is not used.

• Reserved This is 22 bits long.

• Service Parameter Response This is 10 bits long. It contains nine sub-fields.

• TASK RETRY IDENTIFICATION REQUESTED This is 1 bit. When this bit is set to 1, the target confirms support for task retry identification. When this bit is set to 0, the target does not support task retry identification.

• RETRY This is 1 bit. When this bit is set to 1, the target confirms support for retransmission of dropped frames. When this bit is set to 0, the target does not support retransmission of dropped frames.

• CONFIRMED COMPLETION ALLOWED This is 1 bit. When this bit is set to 1, the target declares support for the FCP_CONF IU. When this bit is set to 0, the target does not support the FCP_CONF IU.

• DATA OVERLAY ALLOWED This is 1 bit. This bit is always set to 0 in the LS_ACC. This bit is used only by initiators to declare support for data overlay. If the initiator declares support for data overlay, the target may choose whether to transfer data using random offsets or sequential offsets.

• INITIATOR FUNCTION This is 1 bit. Some devices set this bit to 1, but most set this bit to 0.

• TARGET FUNCTION This is 1 bit. This bit is usually set to 1. If this bit is set to 0, an image pair cannot be established. This bit must be set to 1 if the IPE bit is set to 1.

• OBSOLETE This is 2 bits long. It is not used.

• READ FCP_XFER_RDY DISABLED This is 1 bit. This bit is always set to 1.

• WRITE FCP_XFER_RDY DISABLED This is 1 bit. When this bit is set to 1, the target confirms support for unsolicited data. When this bit is set to 0, the target does not support unsolicited data.

If the PRLI is not valid, the target responds with LS_RJT. A single LS_RJT format is defined in the FC-LS specification for all ELS commands. Figure 8-24 illustrates the format of LS_RJT.

Figure 8-24. LS_RJT ELS Format

A brief description of each field follows:

• LS Command Code This is 8 bits long. It is set to 0x01.

• Unused This is 24 bits long. It is set to 0.

• Reserved This is 8 bits long.

• Reason Code This is 8 bits long. It indicates why the ELS command was rejected. A common set of reason codes is used for all ELS commands. Table 8-14 summarizes the reason codes defined by the FC-LS specification. All reason codes excluded from Table 8-14 are reserved.

  Table 8-14. LS_RJT Reason Codes

  Reason Code	Description
  0x01	Invalid LS Command Code
  0x03	Logical Error
  0x05	Logical Busy
  0x07	Protocol Error
  0x09	Unable To Perform Command Request
  0x0B	Command Not Supported
  0x0E	Command Already In Progress
  0xFF	Vendor Specific Error

  
  

• Reason Explanation This is 8 bits long. It provides additional diagnostic information that complements the Reason Code field. It uses a unique set of reason explanations for each ELS command. Table 8-15 summarizes the reason explanations defined by the FC-LS specification that are relevant to PRLI. All reason explanations excluded from Table 8-15 are either irrelevant to PRLI or reserved.

• Vendor Specific This is 8 bits long. When the Reason Code field is set to 0xFF, this field provides a vendor-specific reason code. When the Reason Code field is set to any value other than 0xFF, this field is ignored.

Table 8-15. LS_RJT Reason Explanations for PRLI

Reason Explanation	Description
0x00	No Additional Explanation
0x1E	PLOGI Required
0x2C	Request Not Supported

The REC ELS enables an initiator to ascertain the state of a given Exchange in a target. Support for REC is optional. When an initiator detects an error (for example, a timeout), the initiator may use REC to determine what, if any, recovery steps are appropriate. Possible responses to REC include LS_ACC and LS_RJT. REC and its associated responses each are encapsulated in the Data field of an FC frame. The fields in the FC Header indicate the payload is a REC, LS_ACC, or LS_RJT. Table 8-16 summarizes the values of the relevant FC Header fields. If command retry is supported, the Parameter field in the FC Header contains the Task Retry Identifier of the Exchange referenced in the REC. If command retry is not supported, the Parameter field in the FC Header is set to 0. The formats of REC and its associated responses are defined in the FC-LS specification. Figure 8-25 illustrates the format of REC.

Table 8-16. FC Header Field Values for REC and LS_ACC ELS

ELS	R_CTL Routing	R_CTL Information Category	Type	F_CTL Relative Offset Present	DF_CTL
REC	0010b	0010b	0x01	0b	0x00 or 0x40
LS_ACC	0010b	0011b	0x01	0b	0x00 or 0x40
LS_RJT	0010b	0011b	0x01	0b	0x00 or 0x40

Figure 8-25. REC ELS Format

A brief description of each field follows:

• LS Command Code This is 8 bits long. It is set to 0x13.

• Unused This is 24 bits long. It is set to 0.

• Reserved This is 8 bits long.

• Exchange Originator S_ID This is 24 bits long. This field contains the FCID of the FC device that originated the Exchange about which state information is sought. Inclusion of this field in the payload enables an FC device to query state information for an Exchange originated by a different FC device. Without this field, the REC recipient would have to rely on the S_ID field in the FC Header, thus disabling third-party queries.

• OX_ID This is 16 bits long. This field contains the OX_ID of the Exchange about which state information is sought. The REC recipient uses this field in combination with the RX_ID field to identify the proper Exchange.

• RX_ID This is 16 bits long. This field contains the RX_ID of the Exchange about which state information is sought. The REC recipient uses this field in combination with the OX_ID field to identify the proper Exchange. If the value of this field is 0xFFFF (unassigned), the REC recipient uses the Exchange Originator S_ID field in combination with the OX_ID field to identify the proper Exchange. If the value of this field is anything other than 0xFFFF, and the REC recipient has state information for more than one active Exchange with the specified OX_ID-RX_ID combination, the REC recipient uses the Exchange Originator S_ID field in combination with the OX_ID and RX_ID fields to identify the proper Exchange.

In the absence of errors, the target responds to REC with LS_ACC. Figure 8-26 illustrates the format of LS_ACC sent in response to REC.

Figure 8-26. LS_ACC ELS Format for REC ELS

A brief description of each field follows:

• LS Command Code This is 8 bits long. It is set to 0x02.

• Unused This is 24 bits long. It is set to zero.

• OX_ID This is 16 bits long. This field contains the OX_ID of the Exchange about which state information is sought.

• RX_ID This is 16 bits long. This field contains the RX_ID of the Exchange about which state information is sought. If the RX_ID specified in the REC request is 0xFFFF, the REC recipient may set this field to the value previously assigned. This situation occurs when a target receives a new command (thus creating state information for the Exchange), but the initial reply is dropped, so the initiator does not know the assigned value of RX_ID.

• Reserved This is 8 bits long.

• Originator Address Identifier This is 24 bits long. This field contains the FCID of the FC device that originated the Exchange about which state information is sought.

• Reserved This is 8 bits long.

• Responder Address Identifier This is 24 bits long. This field contains the FCID of the FC device that responded to the Exchange about which state information is sought. The value of this field might be different than the value of the D_ID field in the FC Header of the REC request if the REC recipient contains more than one FC port.

• FC4VALUE This is 32 bits long. The meaning of this field is defined by each FC-4 protocol specification. The FCP-3 specification defines this field as the total byte count received or sent by the Exchange about which state information is sought. This field can report the byte count for only one direction of transfer. However, the FCP-3 specification does not explicitly prohibit reporting the total byte count for both directions when the REC request references a bidirectional Exchange.

• E_STAT This is 32 bits long. Each FC device maintains state information about each of its active Exchanges in a chunk of memory called the exchange status block (ESB). The ESB is defined in the FC-FS-2 specification. One of the fields in the ESB is called E_STAT. The E_STAT field in the ESB is 4 bytes long. It contains a broad array of information about the Exchange. The E_STAT field of the ESB is encapsulated in the E_STAT field within the LS_ACC ELS.

If the REC is not valid, or if the target does not support REC, the target responds with LS_RJT. Figure 8-24 illustrates the format of LS_RJT. Table 8-14 summarizes the LS_RJT reason codes. Table 8-17 summarizes the reason explanations defined by the FC-LS specification that are relevant to REC. All reason explanations excluded from Table 8-17 are either irrelevant to REC or reserved.

Table 8-17. LS_RJT Reason Explanations for REC

Reason Explanation	Description
0x00	No Additional Explanation
0x15	Invalid Originator S_ID
0x17	Invalid OX_ID-RX_ID Combination
0x1E	PLOGI Required

The ABTS BLS enables an initiator or target to abort a sequence or an entire Exchange. As with all BLS commands, support for ABTS is mandatory. ABTS may be transmitted even if the number of active sequences equals the maximum number of concurrent sequences negotiated during PLOGI. Likewise, ABTS may be transmitted by an FCP device even if it does not hold the sequence initiative. This exception to the sequence initiative rule must be allowed because the sequence initiative is transferred with the last frame of each sequence, but the receiving device neither acknowledges receipt of frames nor notifies the sender of dropped frames (see Chapter 5). Thus, the transmitting device typically detects errors via timeout after transferring the sequence initiative. As a result, ABTS can be sent only by the device that sent the sequence being aborted. Each ABTS is transmitted within the Exchange upon which the ABTS acts. Moreover, the SEQ_ID in the FC Header of the ABTS must match the SEQ_ID of the most recent sequence transmitted by the device that sends the ABTS. In other words, a device may abort only its most recently transmitted sequence or Exchange. The sequence initiative is always transferred with ABTS so the receiving device can respond. The responding device may hold the sequence initiative after responding, or transfer the sequence initiative with the response. Possible responses to an ABTS include the basic accept (BA_ACC) BLS and the basic reject (BA_RJT) BLS. The action taken upon receipt of an ABTS is governed by the Exchange Error Policy in affect for the Exchange impacted by the ABTS. ABTS and its associated responses are each encapsulated in the Data field of an FC frame. The fields in the FC Header indicate that the payload is a BLS command or a BLS response. Table 8-18 summarizes the values of the relevant FC Header fields. Bit 0 in the Parameter field in the FC Header conveys whether the ABTS acts upon a single sequence (set to 1) or an entire Exchange (set to 0). The ABTS does not have a defined format within the Data field of the FC frame. Additionally, the ABTS has no payload. The ABTS recipient uses only the FC Header to determine that a given FC frame is actually an ABTS. The formats of the BA_ACC and BA_RJT are defined in the FC-FS-2 specification. A unique BA_ACC format is defined for each BLS command. Figure 8-27 illustrates the format of the BA_ACC associated with ABTS.

Figure 8-27. BA_ACC Format for ABTS BLS

Table 8-18. FC Header Field Values for ABTS and Responses

BLS	R_CTL Routing	R_CTL Information Category	Type	F_CTL Relative Offset Present	DF_CTL
ABTS	1000b	0001b	0x00	0b	0x00 or 0x40
BA_ACC	1000b	0100b	0x00	0b	0x00 or 0x40
BA_RJT	1000b	0101b	0x00	0b	0x00 or 0x40

A brief description of each field follows:

• SEQ_ID Validity This is 8 bits long. When this field is set to 0x80, the SEQ_ID Of Last Deliverable Sequence field contains a valid SEQ_ID. When this field is set to 0x00, the SEQ_ID Of Last Deliverable Sequence field is ignored.

• SEQ_ID of Last Deliverable Sequence This is 8 bits long. When the SEQ_ID Validity field is set to 0x80, this field contains the SEQ_ID of the last sequence successfully delivered to the SAL. When the SEQ_ID Validity field is set to 0x00, the transmitter does not provide any information about the last deliverable sequence, and the entire Exchange must be aborted.

• Reserved This is 16 bits long.

• OX_ID This is 16 bits long. This field contains the OX_ID of the Exchange upon which the ABTS acts.

• RX_ID This is 16 bits long. This field contains the RX_ID of the Exchange upon which the ABTS acts.

• Low SEQ_CNT This is 16 bits long. When aborting a sequence, this field contains the SEQ_CNT of the last frame of the last sequence successfully delivered to the SAL. When aborting an Exchange, this field is set to 0x0000.

• High SEQ_CNT This is 16 bits long. When aborting a sequence, this field contains the SEQ_CNT of the ABTS frame. When aborting an Exchange, this field is set to 0xFFFF. The range of SEQ_CNT values covered by the Low SEQ_CNT field and the High SEQ_CNT field is combined with the FQXID to generate the Recovery Qualifier. The Recovery Qualifier is valid for a period equal to R_A_TOV. For the transmitter of the ABTS, the R_A_TOV timer begins upon receipt of the BA_ACC. SEQ_CNT values within the Recovery Qualifier range may not be reused while the Recovery Qualifier is valid. Upon expiration of the R_A_TOV timer, the transmitter of the ABTS transmits a Reinstate Recovery Qualifier (RRQ) ELS. The RRQ ELS retires the Recovery Qualifier and enables the reuse of counter values covered by the Recovery Qualifier.

A common BA_RJT format is defined for all BLS commands. Figure 8-28 illustrates the format of the BA_RJT.

Figure 8-28. BA_RJT Format

A brief description of each field follows:

• Reserved This is 8 bits long.

• Reason Code This is 8 bits long. It indicates why the ABTS was rejected. Table 8-19 summarizes the reason codes defined by the FC-FS-2 specification. All reason codes excluded from Table 8-19 are reserved.

Table 8-19. BA_RJT Reason Codes

Reason Code	Description
0x01	Invalid Command Code
0x03	Logical Error
0x05	Logical Busy
0x07	Protocol Error
0x09	Unable To Perform Command Request
0xFF	Vendor Specific Error

• Reason Explanation This is 8 bits long. It provides additional diagnostic information that complements the Reason Code field. Table 8-20 summarizes the reason explanations defined by the FC-FS-2 specification. All reason explanations excluded from Table 8-20 are reserved.

  Table 8-20. BA_RJT Reason Explanations

  Reason Explanation	Description
  0x00	No Additional Explanation
  0x03	Invalid OX_ID-RX_ID Combination
  0x05	Sequence Aborted, No Sequence Information Provided

  
  

• Vendor Specific This is 8 bits long. When the Reason Code field is set to 0xFF, this field provides a vendor-specific reason code. When the Reason Code field is set to any value other than 0xFF, this field is ignored.

The SRR link service enables an initiator to request retransmission of data during a read command, request that the target request retransmission of data during a write command, or request retransmission of the FCP_RSP IU during a read or write command. Only initiators may send an SRR. Support for SRR is optional. If SRR is supported by both initiator and target, REC and task retry identification also must be supported by both devices. The Parameter field in the FC Header contains the Task Retry Identifier of the Exchange referenced by the SRR payload. SRR may not be used during bidirectional commands. This limitation does not apply to iSCSI, which supports retransmission during bidirectional commands. The sequence initiative of the Exchange referenced by the SRR is always transferred to the target upon receipt of an SRR. This allows the target to transmit the requested IU. The sequence initiative of the SRR Exchange is also transferred to the target. Possible responses to SRR include the SRR Accept and the FCP Reject (FCP_RJT). The SRR and its associated responses are each encapsulated in the Data field of an FC frame. The fields in the FC Header indicate that the payload is an FC-4 link service request or an FC-4 link service response. Table 8-21 summarizes the values of the relevant FC Header fields. The formats of SRR and its associated responses are defined in the FCP-3 specification. Figure 8-29 illustrates the format of SRR.

Table 8-21. FC Header Field Values for SRR and Responses

FC-4 LS	R_CTL Routing	R_CTL Information Category	Type	F_CTL Relative Offset Present	DF_CTL
SRR	0011b	0010b	0x08	0b	0x00 or 0x40
SRR Accept	0011b	0011b	0x08	0b	0x00 or 0x40
FCP_RJT	0011b	0011b	0x08	0b	0x00 or 0x40

Figure 8-29. SRR Link Service Format

A brief description of each field follows:

• FC-4 Link Service Command Code This is 32 bits long. It is set to 0x14000000.

• OX_ID This is 16 bits long. This field contains the OX_ID of the Exchange for which retransmission is requested.

• RX_ID This is 16 bits long. This field contains the RX_ID of the Exchange for which retransmission is requested.

• Relative Offset This is 32 bits long. When an initiator requests retransmission of an FCP_RSP IU, this field is ignored. When an initiator requests retransmission of data or requests that the target request retransmission of data, this field contains the offset of the lowest numbered byte that needs to be retransmitted. The two low-order bits in this field are always set to 0, so the requested data always begins on a 4-byte word boundary. Note that the SRR link service does not request retransmission of one or more specific IUs. Instead, data is requested based on the relative offset provided in this field. This contrasts with the iSCSI model. In theory, the relative offset approach constitutes a more efficient recovery mechanism because it enables the requestor to specify only those data bytes that need to be retransmitted. However, the SRR link service lacks the ability to specify the upper boundary of data to be retransmitted. Thus, all data bytes with a relative offset equal to or greater than the value specified in this field must be retransmitted. This limitation mitigates the data retransmission efficiencies that FCP could realize compared to iSCSI.

• R_CTL for IU This is 8 bits long. This field specifies the type of IU being requested. This field contains one of the values defined in the FC-FS-2 specification for the R_CTL field in the FC Header. Valid values are 0x01 (solicited data), 0x05 (data descriptor) and 0x07 (command status).

• Reserved This is 24 bits long.

In the absence of errors, the target responds to SRR with SRR Accept. Figure 8-30 illustrates the format of SRR Accept.

Figure 8-30. SRR Accept Format

A brief description of the field follows:

• FC-4 Link Service Command Code This is 32 bits long. It is set to 0x02000000.

Upon receipt of an FCP_RJT, the initiator must abort the entire Exchange referenced by the SRR. Figure 8-31 illustrates the format of FCP_RJT.

Figure 8-31. FCP_RJT Format

A brief description of each field follows:

• FC-4 Link Service Command Code This is 32 bits long. It is set to 0x01000000.

• Reserved This is 8 bits long.

• Reason Code This is 8 bits long. It indicates why the SRR link service was rejected. Table 8-22 summarizes the reason codes defined by the FCP-3 specification. All reason codes excluded from Table 8-22 are reserved.

• Reason Explanation This is 8 bits long. It provides additional diagnostic information that complements the Reason Code field. Table 8-23 summarizes the reason explanations defined by the FCP-3 specification. All reason explanations excluded from Table 8-23 are reserved.

• Vendor Specific This is 8 bits long. When the Reason Code field is set to 0xFF, this field provides a vendor specific reason code. When the Reason Code field is set to any value other than 0xFF, this field is ignored.

Table 8-22. FCP_RJT Reason Codes

Reason Code	Description
0x01	Invalid FCP FC-4 Link Service Command Code
0x03	Logical Error
0x05	Logical Busy
0x07	Protocol Error
0x09	Unable To Perform Command Request
0x0B	Command Not Supported
0xFF	Vendor Specific Error

Table 8-23. FCP_RJT Reason Explanations

Reason Explanation	Description
0x00	No Additional Explanation
0x03	Invalid OX_ID-RX_ID Combination
0x2A	Unable To Supply Requested Data

The preceding discussion of FCP IU and FC link service formats is simplified for the sake of clarity. Comprehensive exploration of FCP IU and FC link service usage is outside the scope of this book. For more information, readers are encouraged to consult the ANSI T10 SAM-3, SAM-4, and FCP-3 specifications, and the ANSI T11 FC-FS-2 and FC-LS specifications.

FCP Additional Login Parameters

While most FCP service parameters are negotiated or declared via the PRLI mechanism described in the preceding section, some FCP service parameters are negotiated via the SCSI MODE SENSE and MODE SELECT commands. Detailed exploration of the SCSI MODE commands is outside the scope of this book, but readers should at least be aware of the service parameters that FCP negotiates via the SCSI MODE commands. So, this section provides a brief description of the service parameters relevant to FCP that are negotiated via the SCSI MODE commands. Three SCSI mode pages are relevant to FCP service parameter negotiation. They are the Disconnect-Reconnect mode page, the Protocol Specific Logical Unit mode page, and the Protocol Specific Port mode page.

The Disconnect-Reconnect mode page is used to optimize performance between an initiator port and a target port. Three parameters are relevant to fabric-attached FCP devices: Maximum Burst Size, First Burst Size, and Data Overlay. The Maximum Burst Size is the maximum number of bytes that may be transferred in a single FCP_DATA IU. Because FCP mandates a one-to-one relationship between IUs and sequences, this parameter also determines the maximum number of bytes that may be transferred in a single sequence. However, this parameter might not equate to the maximum number of bytes transferred during a single possession of the sequence initiative, because sequence streaming is allowed. The First Burst Size is the maximum number of bytes that may be transferred in a single unsolicited FCP_DATA IU. This parameter is ignored if the WRITE FCP_XFER_RDY DISABLED bit in the PRLI Request or associated LS_ACC ELS is set to 0. Data overlay is negotiated via the enable modify data pointers (EMDP) bit. The EMDP bit overrides the value of the DATA OVERLAY ALLOWED bit in the PRLI Request.

The Protocol Specific Logical Unit mode page as implemented by FCP is called the FC Logical Unit Control mode page. The FC Logical Unit Control mode page is used to configure certain logical unit service parameters. In particular, FCP implements the enable precise delivery checking (EPDC) bit. The EPDC bit enables or disables in-order command delivery (called precise delivery in FCP parlance). Unlike iSCSI, FCP does not mandate precise delivery. Instead, FCP allows precise delivery to be negotiated with each logical unit.

The Protocol Specific Port mode page as implemented by FCP is called the FC Port Control mode page. The FC Port Control mode page is used to configure certain target port service parameters. For fabric-attached devices, FCP implements the sequence initiative resource recovery time-out value (RR_TOV_{SEQ_INIT}). This timer determines the minimum amount of time a target port must wait for a response after transferring the sequence initiative. If this timer expires before a response is received, the target port may begin error recovery procedures. For more information about FCP's use of the SCSI MODE commands, readers are encouraged to consult the ANSI T10 SPC-3 and FCP-3 specifications.

FCP Delivery Mechanisms

FCP leverages the delivery mechanisms implemented by FC. Additionally, the FCP-3 specification defines several mechanisms for detecting and recovering from errors. Like iSCSI, FCP supports error recovery at the task (Exchange) level and the IU (sequence) level. In other words, an FCP device may choose to abort each task affected by an error (suboptimal) or retransmit just the dropped IUs (optimal). The choice is subject to the Exchange Error Policy in effect for the Exchange affected by the error. Like iSCSI, FCP does not support frame level error recovery. However, iSCSI benefits from frame level retransmission as implemented by TCP, whereas FCP does not because FC does not support frame level retransmission (see Chapter 5).

Error Detection

Like iSCSI, FCP mandates that both initiators and targets are responsible for detecting errors. Errors are detected via timeouts, FC primitives, FC Header fields, FCP IU fields, the ABTS BLS, and the REC ELS. Initiators and targets must be able to detect the link errors, protocol errors, and timeouts defined in the FC specifications. Additionally, initiators and targets must be able to detect protocol errors and timeouts using FCP IU fields. Last, initiators and targets must be able to accept and process the ABTS BLS as a means of error detection. The only error detection mechanism that is optional is the REC ELS.

Exchange Level Error Recovery

The initiator is responsible for performing error recovery, but the target may optionally invoke recovery action under certain conditions. At the Exchange level, the error recovery procedure is called Recovery Abort. Recovery Abort is defined in the FC-FS-2 specification. The

FCP-3 specification defines rules for using Recovery Abort. Recovery Abort is used to abort an Exchange and to recover the associated resources within the initiator and target devices. All initiators must be able to invoke Recovery Abort. All targets must be able to execute Recovery Abort in response to initiator invocation. Recovery Abort may be invoked in response to an error or by a TMF request. When Recovery Abort is invoked, the SCSI application client must reissue the aborted SCSI task.

If the target detects a frame error, the target discards all frames associated with the Exchange. The target takes no further action and waits for the initiator to detect the error via timeout. Upon detecting an error (via timeout or otherwise), the initiator optionally sends a REC to the target. If the REC confirms that an error has occurred, or if the initiator chose not to send a REC, the initiator sends an ABTS to the target. Bit 0 in the Parameter field in the FC Header is set to 0 indicating that the scope of the ABTS is the entire Exchange. The target takes the necessary internal actions to ensure the affected logical unit is notified. The target then responds with a BA_ACC. The Last_Sequence bit in the F_CTL field in the FC Header of the BA_ACC must be set to 1 (to terminate the Exchange). Upon receiving the BA_ACC, the initiator begins the R_A_TOV timer and notifies the SCSI application client. The SCSI application client may then reissue the command. Upon expiration of the R_A_TOV timer, the initiator sends an RRQ to the target. The RRQ enables reuse of the counter values covered by the Recovery Qualifier. When the initiator receives an LS_ACC in response to the RRQ, the Recovery Abort procedure is complete.

If the initiator does not receive an LS_ACC in response to the REC within two times the R_A_TOV, the REC exchange is aborted via Recovery Abort. The REC may be retried in a new exchange, or the initiator may choose to send an ABTS without retrying REC. If the initiator does not receive a BA_ACC within two times the R_A_TOV, the initiator may send another ABTS. If the second BA_ACC is not received within two times the R_A_TOV, the initiator must explicitly logout the target. The initiator must then perform PLOGI and PRLI to continue SCSI operations with the target, and all previously active SCSI tasks must be reissued by the SCSI application client.

Sequence Level Error Recovery

Initiators and targets are not required to support sequence level error recovery. Sequence level error recovery involves the abortion of a single sequence and the retransmission of the missing data. Upon detecting an error (via timeout or otherwise), the initiator optionally sends a REC to the target. If the REC confirms an error has occurred, or if the initiator chose not to send a REC, the initiator sends an ABTS to the target. Bit 0 in the Parameter field in the FC Header is set to 1, indicating that the scope of the ABTS is a single sequence. The target discards all frames associated with the sequence identified by the ABTS. The target then responds with a BA_ACC. The Last_Sequence bit in the F_CTL field in the FC Header must be set to 0. Upon receiving the BA_ACC, the initiator sends an SRR. Upon receiving the SRR, the target responds with an SRR Accept. The target then retransmits the requested data. In the case of a write command, the SRR requests an FCP_XFER_RDY IU that requests the missing data. The SEQ_CNT field in the FC Header of the first frame of retransmitted data is set to 0 even if continuously increasing SEQ_CNT is in affect for the connection. Note the retransmitted data can be (and often is) transferred from a relative offset that was already used in the Exchange being recovered. This is allowed even if the DATA OVERLAY ALLOWED bit is set to 0 during PRLI.

If the initiator does not receive an LS_ACC in response to the REC within two times the R_A_TOV, the REC exchange is aborted via Recovery Abort. The REC may be retried in a new exchange, or the initiator may choose to send an ABTS without retrying REC. If the initiator does not receive a BA_ACC within two times the R_A_TOV, the initiator may send another ABTS. If the second BA_ACC is not received within two times the R_A_TOV, the initiator must explicitly logout the target. The initiator must then perform PLOGI and PRLI to continue SCSI operations with the target, and all previously active SCSI tasks must be reissued by the SCSI application client. If a BA_ACC is received, but the initiator does not receive an SRR Accept within two times the R_A_TOV, both the SRR Exchange and the Exchange being recovered must be aborted via Recovery Abort.

FCP IU Boundary Detection

Because each FCP IU maps to a single sequence, IU boundaries can be easily detected via the SEQ_ID field in the FC Header. The SEQ_ID field is present in the FC Header of every FC frame, so IU boundaries can be detected even when one or more frames are dropped. So, FCP does not require a message synchronization scheme. This contrasts with the iSCSI model.

FCP In-Order Command Delivery

Like iSCSI, FCP supports in-order command delivery (precise delivery). Unlike iSCSI, precise delivery is optional for FCP devices. Another key difference is the scope of enforcement. Whereas iSCSI enforces in-order command delivery for all commands between each initiator-target pair, FCP enforces precise delivery on a per-LUN basis. The initiator sends an FC Logical Unit Control mode page to each LUN that is discovered by the report luns command. For each LUN that responds with the EPDC bit set to 1, commands are delivered in-order. In practice, the enforcement difference is purely academic because most modern storage arrays support precise delivery on all LUNs for performance reasons. However, the per-LUN nature of precise delivery has a real effect on the way FCP implements command numbering. A separate CRN counter is maintained for each LUN that supports precise delivery. In other words, a CRN counter is implemented per I_T_L nexus.

Note

The order of command delivery does not necessarily translate to the order of command execution. The order of command execution can be changed via TMF request as specified in the SCSI standards.

The CRN field is set to 0 in every FCP_CMND IU that conveys a TMF request regardless of whether the LUN supports precise delivery. Likewise, the CRN field is set to 0 in every FCP_CMND IU that conveys a SCSI command to LUNs that do not support precise delivery. By contrast, the CRN field contains a value between 1 and 255 in each FCP_CMND IU that conveys a SCSI command to LUNs that support precise delivery. The CRN counter for a given LUN is incremented by 1 for each command issued to that LUN. However, certain SCSI commands (such as inquiry and test unit ready) do not require precise delivery and may be assigned a value of 0 for the CRN field even when issued to LUNs that support precise delivery. When the CRN counter reaches 255, the value wraps back to one for the next command. The limited number range of the CRN counter exposes FCP devices to possible ambiguity among commands in environments that support sequence level error recovery. State information about each Exchange and sequence must be maintained for specific periods of time to support retransmission. Eventually, the initiator will issue a new command using an OX_ID that was previously used within the session. If the target is still maintaining state for the old command associated with the reused OX_ID, the target could confuse the two commands. To mitigate this risk, task retry identification must be implemented. The Task Retry Identifier effectively extends the CRN counter by providing an additional identifier for each command. The Task Retry Identifier must be consistent in all IUs and link services related to a single command (FCP_CMND, REC, SRR).

Note

In high I/O environments, the limited CRN counter might seem to pose a risk to performance. However, each LUN uses a separate CRN counter, which effectively mitigates any risk to performance by increasing the total CRN address space. In the unlikely event that an initiator needs to issue more than 255 ordered commands to a single LUN before the first command completes, the CRN counter limit would prevent issuance of the 256^th command. However, when so many commands are simultaneously outstanding, it indicates that the LUN cannot execute commands as quickly as the application requires. Thus, the LUN is the performance bottleneck, not the CRN counter.

FCP Connection and Session Recovery

FCP does not support stateful connection or session recovery. If a connection fails, FCP notifies the SAL of the I_T nexus loss, and all state related to the connection and session is cleared. The initiator must perform PLOGI and PRLI to continue SCSI operations with the target, and all previously active SCSI tasks must be reissued by the SCSI application client. If a session fails, FCP notifies the SAL of the I_T nexus loss, and all state related to the session is cleared. The initiator must perform PRLI to continue SCSI operations with the target, and all previously active SCSI tasks must be reissued by the SCSI application client.

The preceding discussion of FCP delivery mechanisms is simplified for the sake of clarity. For more information about FCP delivery mechanisms, readers are encouraged to consult Chapter 5 and the ANSI T10 FCP-3 specification.