2.6 Storage Area Networks
Entire books have been written about SANs. Two notable
Achieving the roadmap objectives of
IP Storage Networking ”Straight to the
2.6.1 SAN History and Background
In the mid-1990s, IT professionals recognized the gap between DAS and NAS, and momentum grew for a mechanism that combined the performance and direct access of DAS with the network flexibility of NAS. The solution that emerged was Fibre Channel.
Fibre Channel is actually a SCSI-based implementation, only in a serial format. SCSI commands are at the root of all storage exchanges between imitators and targets. Commands such as write, read, and
Originally, the SCSI command set could
Figure 2-9. Basics of Fibre Channel Protocol (FCP).
With the routable packaging in place via FCP shown in Figure 2-9, the network build-out for storage could take place with Fibre Channel SANs, shown in Figure 2-10.
Figure 2-10. Fibre Channel storage area networks.
Fibre Channel SANs proved invaluable for storage professionals looking to expand direct access storage capacity and utilization. However, Fibre Channel SANs came with their own set of issues that left the end-to-end solution incomplete. This includes
Fibre Channel SANs, through the inherent nature of introducing a new network technology different from IP and Ethernet, requires new staff training, new equipment, and new management systems. These factors add considerably to the overall total cost of the storage infrastructure.
Finally, Fibre Channel networking was designed as a data center technology, not intended to travel long distances across WANs. With today's requirements for high-availability across geographic areas, Fibre Channel networks cannot
2.6.2 Pure IP Storage Networks
Starting in 2000, the concept of a pure IP SAN took hold. The basic concept of this implementation was to create a method of connecting servers and storage devices, with the direct access block
Figure 2-11. Basics of Internet Small Computer System Interface (iSCSI).
iSCSI, in its simplest form, represents a mechanism to take block-oriented SCSI commands and map them to an IP network. This protocol can be implemented directly on servers or storage devices to allow native connectivity to IP and Ethernet networks or fabrics. Once accomplished, storage applications requiring direct access to storage devices can extend that connection across an IP and Ethernet network, which has a nearly ubiquitous footprint across corporate and carrier infrastructures. The basics of the iSCSI protocol are outlined in Figure 2-11, while an implementation example is shown in Figure 2-12.
Figure 2-12. Pure iSCSI storage networks.
This wholehearted embrace of underlying IP and Ethernet technologies ”mature network technology, installed base, software integration, industry research ”should guarantee iSCSI will succeed as a storage transport. However, networking technologies succeed based on provision of a transition path. The same applies for iSCSI, where the transition
Adoption of new technologies occurs most frequently with those likely to gain. With iSCSI, that would
For a pure iSCSI storage network, no accommodation is made for Fibre Channel. More
Several mechanisms exist to enhance current Fibre Channel SANs with IP. Optimized storage deployment depends on careful analysis of these solutions, attention to control points within the architecture, and recognition that aggressively embracing IP networking can lead to dramatic total cost benefits. These considerations are covered in Section 2.6.5, "Enhancing Fibre Channel SANs with IP."
2.6.3 SAN GUIDANCE: Comparing Ethernet/IP and Fibre Channel Fabrics
With the introduction of IP storage as a capable storage network transport, many comparisons have been drawn between the two technologies. For fair comparison, one must recognize that Fibre Channel and Ethernet are both Layer 2 networking technologies virtually equivalent in performance. They both provide point-to-point,
However, Ethernet is the largest home for the Internet Protocol, and IP ubiquity dramatically expands fabric flexibility. The equivalent in Fibre Channel would be the FCP, but since that works only on Fibre Channel fabrics, it can't compete with IP as an
Fibre Channel SANs function well in data center environments and are useful for massive Fibre Channel end-device consolidation. But for reasons identified above, Fibre Channel requires complementary technologies such as IP and Ethernet to help integrate storage into overall corporate cost and management objectives.
Table 2-1. Comparison of Fibre Channel and Ethernet/IP Fabrics
2.6.4 Host Bus Adapters and Network Interface Cards
Regardless of the networking interconnect, servers and storage devices need methods to take data from the CPU or the disk
With the dramatic differences between traditional IP-based LAN traffic using NICs and traditional
Fibre Channel-based HBAs compensated for this inefficiency by moving the data segmentation directly to the card and letting the card do the heavy
A combination of these two approaches results in the efficiency of the Fibre Channel approach, through a process called TCP/IP offload. TCP/IP offload can be used for network based servers using TCP/IP, including iSCSI or NAS. The processing for data segmentation once completed in the host now moves to the card, freeing CPU cycles. Also, the data comes out of the server directly to a TCP/IP and Ethernet link using iSCSI, providing native access to IP networks and the corresponding universal reach. IP storage can run on traditional IP NICs through software drivers, but the CPU utilization will be high and could impact application performance during operating hours. During off-peak times, when CPU cycles are available, this approach may be sufficient.
IP storage adapters can be further categorized by the types of offload mechanisms. At the basic level, all IP storage adapters
Figure 2-13. Network interface cards, host bus adapters, IP storage adapters.
2.6.5 Enhancing Fibre Channel SANs with IP
Critical decisions face storage and networking professionals as they try to balance Fibre Channel storage and SAN investment with IP networking investment. The buildup of two incompatible switched, full-duplex, point-to-point networking architectures simply cannot continue long
Traditionally, there has been a distinct separation between networking groups and storage groups within corporate IT departments. Some storage groups have claimed that networking technologies cannot support their requirements and that independent storage traffic requires special networks. The reality of the situation is that IP and Ethernet technologies have supported storage traffic for some time (in file-level formats), and recent innovations such as iSCSI provide block-level transport as well. Going forward, networking professionals interested in an overall view of the IT infrastructure will benefit from a closer look at storage, and vice versa for storage professionals.
Whether or not the merging of Fibre Channel and IP technologies meets certain organizational structures is largely irrelevant. Market forces behind the integration of these technologies are upon us, and savvy technologists and business
2.6.6 Making Use of the IP Core
Enhancing Fibre Channel solutions with IP starts through an understanding of the IP core. This part of the network provides the most performance, scalability, and intelligence to facilitate the Internet-working of any devices. Equipment used in the IP core can include routers, chassis-based enterprise Ethernet switches, and optical networking equipment using standard Ethernet or DWDM. Other networking technologies include ATM, Frame Relay, and SONET, all of which have mature mechanisms for handling IP transport.
At the edge of the network reside Fibre Channel devices, including servers, storage, and Fibre Channel SAN switches and directors. This equipment provides access to Fibre Channel devices, specifically end-device aggregation, and the fan-out of storage ports.
In order to make use of the existing IP core, which
Figure 2-14. Using IP to enhance Fibre Channel.
2.6.7 Linking Fibre Channel to Fibre Channel with IP
The most common initial deployments for Fibre Channel using IP include remote replication for large Fibre Channel disk subsystems and linking two Fibre Channel SANs across IP,
Similar architectures exist that may also include Fibre Channel SANs, such as switches and directors. Those Fibre Channel SAN islands can also connect via IP using IP storage switches or gateways. Both of these options are shown in Figure 2-15. Servers and storage within each SAN can be assigned to view other devices locally as well as across the IP connection.
Figure 2-15. Mechanisms for linking Fibre Channel SANs and devices with IP networks.
The benefits of these architectures include the ability to use readily available and
2.6.8 Integrating iSCSI with Fibre Channel across IP
The addition of iSCSI devices means that more of the connection between initiators and targets can be IP-centric. Storage planners gain tremendous flexibility and management advantages through this option, maximizing the use of IP networking and localizing the Fibre Channel component to the attachment of storage devices.
Using iSCSI devices with native IP interfaces allows the devices to talk to each other across any IP and Ethernet network. However, iSCSI makes no provision for Fibre Channel, and the conversion between iSCSI devices and Fibre Channel devices must reside within the network. This protocol conversion process typically takes place within an IP storage switch or gateway that supports both iSCSI and Fibre Channel. It can also take place in a server with specialized software or even within a chip that is embedded in a storage subsystem, as shown in Figure 2-16.
Figure 2-16. Mechanisms for linking Fibre Channel SANs to iSCSI with IP Networks.
For a single device-to-device session, such as server to disk, most multiprotocol products can easily accomplish this task. However, when considering more sophisticated storage device-layer interaction, such as device-to-device interconnect for remote mirroring or SAN extension/interconnect, the protocol conversion process may have more impact on the overall operation. Since iSCSI and Fibre Channel have separate methods of SCSI command encapsulation, the various "states" of each storage session
Figure 2-17. iSCSI and iFCP/FCIP to connect two Fibre Channel end nodes.