Going the Distance with Storage Data

For the past few years companies have been acquiring storage area networks to satisfy specific application requirements. In a single enterprise, separate SANs have appeared in different departments or regional locations as managers have attempted to address the high availability, resource consolidation or tape backup requirements of their own areas. With this initial deployment of SAN islands in various business units, little thought was given to how they might be joined together to streamline shared storage administration and maximize utilization of shared storage resources throughout the enterprise. Now, however, more customers are seeking means to join previously independent SANs together, often across campus, metropolitan or wide area distances. At the same time, customers may want to leverage storage over wide areas for disaster recovery, content distribution or remote tape vaulting applications. Finding a means to efficiently and economically move block storage data across long distances is thus becoming a top priority for many customers, both to satisfy budget requirements and to insure business continuance in the event of disruption or disaster.

Since current SANs are based primarily on Fibre Channel, extending storage data over distance must accommodate the peculiar requirements of Fibre Channel fabrics. Fibre Channel switches are designed for the narrow circumference of the data center or department, and so are not easily adapted to wide area connections. Connecting SAN islands over dark fiber or dense wave division multiplexing (DWDM) may work well within metropolitan environments, but the lack of sufficient buffering in fabric switches results in dramatic loss in performance beyond 50 100 miles. This problem can be addressed by using special IP tunneling gateways that use Fibre Channel over IP and typically provide much larger buffers than stand-alone fabric switches.

In both of these instances, however, the connection between fabric switches over distance creates one extended SAN. This is analogous to a link layer LAN of the 1980s, before the introduction of IP routers. Just as link layer LANs were vulnerable to broadcast storms by errant end devices, an extended SAN may be vulnerable to disruptions in the form of fabric reconvergence or state change notification broadcasts. Problems at one site can be propagated to another site, causing the extended SAN to divide again into separate SAN islands and the loss of storage connectivity.

To overcome this potential issue, the Internet Fibre Channel Protocol (iFCP) was developed. The iFCP protocol allows Fibre Channel SANs to be connected over long distances, but without building a single logical SAN. Instead, the autonomy of each SAN location is maintained, while authorized sessions between servers and storage across the wide area can be established. Disruptions at one site are not propagated across the wide area link, so other sites are unaffected.

While iFCP addresses the issue of stability for SAN extension, basic laws of physics must also be considered. Speed of light propagation delay is roughly a millisecond of latency for every 100 miles of distance. A thousand mile link between two locations, therefore, would incur about 20 milliseconds of latency round trip. Having enormous port buffers in an IP storage gateway is one way to deal with speed of light latency, particularly for streaming storage applications such as tape backup. Instead of having to periodically wait for limited buffers to be replenished, large buffers can be used to fill the wide area link with storage data as acknowledgments return from the receiving site. Additional mechanisms such as data compression, large (jumbo) IP packets, and manipulation of write operations contribute to the more efficient utilization of available wide area bandwidth.

With technology available today it is possible to span thousands of miles of distance with storage data using IP and switched optical network infrastructures. This enables more resilient disaster recovery strategies that place a backup site well beyond a potential disruption while leveraging more affordable IP network services. In addition, an enterprise with national or international storage requirements can now implement comprehensive solutions that were not possible a few years ago. Storage data can now go the distance by leveraging the best of Fibre Channel storage with the best of IP storage switch technology.