Redundant IO Path Elements

Redundant I/O Path Elements

The concept of the I/O path was introduced in Chapter 3, "Getting Down with Storage I/O," as a way to understand the complete picture of how data is transferred between systems and storage. Redundancy can be applied to all the physical components of the path, including

• Host server systems

• HBAs and cables

• Network equipment and routes

• Subsystem ports

• Subsystem interconnects

We'll look at each of these in the following sections.

Redundant Server Systems

Infrastructures for data access necessarily include options for redundant server systems. While server systems might not necessarily be thought of as storage elements, they are clearly key pieces of data access infrastructures. From a storage I/O perspective, servers are the starting point for most data access. Server filing systems are clearly in the domain of storage. To a large degree, network attached storage (NAS) systems are primarily designed around the filing functionality they provide. Chapter 15, "Network File Systems and Network Attached Storage," discusses NAS technology in more detail.

Redundant server systems and filing systems are created through one of two approaches: clustered systems or server farms. Farms are loosely coupled individual systems that have common access to shared data, and clusters are tightly coupled systems that function as a single, fault-tolerant system. If a server fails in a farm, the application job has to be restarted on another server. If a server fails in a cluster, the application job can continue to run with minimal interruptions on another system in the cluster. The topic of clustered and distributed filing systems is explored further in Chapter 16, "New Directions in Network Filing: Clustered File Systems, Distributed File Systems, and Network Storage for Databases."

NOTE

It is certainly possible that redundant servers in server farms can eliminate the need for redundant I/O paths in those servers. If a client/server application is stateless and the client can reconnect with any available server in the farm and access data, there is no need to provide redundant I/O paths on every server. Redundant data access is simply provided by the farm itself.

Redundant HBAs, HBA Ports, and Cables

Multipathing software depends on having redundant I/O paths between systems and storage. In general, changing an I/O path involves changing the initiator used for I/O transmissions and, by extension, all downstream connections. This includes switches and network cables that are being used to transfer I/O data between a computer and its storage.

A single multiported HBA can have two or more ports connecting to the SAN that can be used by multipathing software. However, while multiported HBAs provide path redundancy, most current multipathing implementations use dual HBAs to provide redundancy for the HBA adapter.

Passive and active cabling components can fail or be damaged, effectively shutting down communications. Cables and connectors don't fail often, but they can be cut or physically damaged, resulting in the loss of communications. Electro-optical transceivers are less likely to suffer damage, but they have been known to fail and block communications. Redundant cabling is a natural result of using redundant ports on network nodes and switches.

Network Switching Equipment

Switches (including storage directors) can fail also. Most SANs have been designed with redundant switches to keep a switch failure from occurring and stopping SAN communications. Many possible switch topologies can provide high availability, but that discussion is beyond the scope of this book.

Switches are typically made with redundant, hot-swappable power supplies that allow the switch to continue running after a power supply fails and continue running while a replacement is installed and made active.

SAN Directors are high-availability SAN switches with 64 or more ports and redundant supervisor modules that control the operations of the director. If one supervisor fails, the other assumes control of all director operations until the failed unit is removed and replaced.

In addition to the hardware in a switch, there are also software modules in a switch that manage its operations. The switch control software in a director with dual supervisors can typically be upgraded, one supervisor at a time, without impacting the director's availability.

An important function of SAN switches is building routing tables that determine the path traffic takes through a multiswitch SAN. There can be multiple routes through a network of more than one switch. Network routing and storage multipathing should not be confusedthey are completely different functions. Routing determines which switches traffic is forwarded through, and multipathing determines which initiators and logical unit numbers (LUNs) are to be used.

Subsystem Ports

Storage subsystems, the topic of Chapter 5, "Storage Subsystems," can have multiple network ports connecting to switches in the SAN. The ability to direct storage traffic through different subsystem ports is a critical component of multipathing. Typically one host system HBA is configured to communicate with one subsystem port while the other subsystem HBA is configured to use the other subsystem port. A considerable part of this chapter discusses the subsystem connections and LUN assignments needed to make multipathing work.

Subsystem Interconnects

Although it is not necessarily visible to host system initiators, the storage interconnect used inside a subsystem can have an important impact on the subsystem's ability to provide multipathing support. For instance, a subsystem with a single interconnect has some level of risk for that interconnect failing and making data unavailable. This is one of the reasons storage subsystems tend to use backplanes for connecting devicesto eliminate most of the physical exposures that could stop internal communications.

More important than the physical interconnect technology is the design of the controllers and cache in the subsystem. Subsystem controllers may have two or more multiple interconnect controllers to provide redundancy should one of them fail.