Virtualization Products: Volume Managers and SAN Virtualization Systems

Two basic types of virtualization products are analyzed in this chaptervolume management software and SAN virtualization systems. SAN virtualization systems can be further classified as using either an in-band approach or an out-of-band approach.

NOTE

Storage virtualization technology is the heart and soul of storage subsystem controllers. However, while storage subsystem controllers tend to have extremely powerful virtualization capabilities, they are not often referred to as being "virtualization" products, per se.

Yes, this is another one of those confounding areas where the language of storage makes things more difficult and convoluted than they need to be. The term storage virtualization is used in this chapter to identify the fundamental storing-level functions provided in volume managers, subsystem controllers, and SAN virtualization systems. The term SAN virtualization system refers to the implementation of virtualization in a system that resides in the SAN.

Volume Management

Volume management is a generic name for host system software that manages storage address spaces for systems. It is sometimes referred to as disk management software or software RAID. Disk partitioning software can be thought of as a very simple form of volume management.

Volume management implements storage virtualization functions that interact with the system kernel as a low-level I/O process. It runs between the storing level host bus adapter (HBA) device drivers and filing systems in the I/O stack, which is why volume management is sometimes referred to as a shim layer. When the volume manager receives a request from a filing system for an I/O operation on certain blocks in a storage address space, it translates that request into the multiple address spaces that it is managing. For example, if the volume manager is providing mirroring on two storage address spaces, a single I/O request is processed by the volume manager, which passes two requests to lower-level storage device drivers. If the volume manager is striping data with RAID, the volume manager determines how to segment the original request into multiple requests that are turned into storage commands by lower-level storage device drivers. Figure 12-2 shows where volume manager processes run in relation to the filing system storage device drivers.

Figure 12-2. Volume Management Runs Between the Filing System and Storage Device Drivers

Volume management software became popular as open-systems computers started to scale in size and need larger capacities and higher data availability. Veritas Corporation developed a volume management product for Sun systems that was widely deployed by Sun customers. Volume management was originally developed for parallel SCSI storage devices and subsystems, many of which were relatively inexpensive JBOD subsystems.

Volume managers have adapted well to SAN environments by being able to incorporate a wide variety of storage products as resources for host systems. However, as products that were traditionally licensed and installed on individual systems, traditional volume manager designs do not inherently provide centralized control of storage in a SAN. While multiple volume managers may be able to be managed centrally, that does not mean that all their combined storage resources can be managed centrally as a shared resource.

SAN Virtualization Systems

By contrast, SAN virtualization systems are separate, external implementations of storage virtualization that run in the I/O path between host systems and storage subsystems. They can be thought of as processors of SCSI storage transmissions. They have SAN ports with LUNs that direct storage I/Os to LU processes that manage communications with host initiators. In addition, SAN virtualization systems also act as SCSI initiators for commands that are transmitted to downstream storage subsystems.

The I/O process through a SAN virtualization system begins with an incoming SCSI command from a host initiator that is transmitted to a target controller/LUN and processed by the associated LU in the virtualization system. Each SCSI command is interpreted by controller logic running in the virtualization system in the order it is received and is sent back out over the SAN through the appropriate initiator/port to the downstream storage subsystem.

One way to think about SAN virtualization is that it moves the storage controller function out of a subsystem and locates it in a network. A key architectural difference is that SCSI communications with host systems are managed by LUs in the virtualization system and not by LUs in the storage subsystem. This LU can be thought of as a proxy LU because it does not act directly with devices and media. Similarly, communications with downstream storage subsystem LUs are started by initiators in the virtualization system that can be thought of as proxy initiators. The initiator/LU connections then assume the following form:

host system initiator to proxy LU + proxy initiator to storage LU

Proxy LUs in virtualization systems that manage host SCSI communications are sometimes said to be terminating the I/O. In the same vein, proxy initiators reinitiate the I/O and send it to downstream storage. Transmission to downstream storage often involves several separate I/Os to separate storage subsystems or LUs, depending on the type of address space manipulation being performed by the virtualization system. For instance, a virtualization system that implements mirroring would forward two separate downstream I/Os for every I/O it receives.

Figure 12-3 shows a SAN virtualization system in the I/O path. In this figure, the host initiator sends an original I/O to the proxy LU in the virtualization system, and a proxy initiator in the virtualization system sends a downstream I/O to a real LU in the storage subsystem. This vir-tualization system is not performing any storage address space translations but is serving a simple proxy role.

Figure 12-3. I/O Communications Through a SAN Virtualization System in the I/O Path

Whereas volume management software typically runs in a single system, SAN virtualization systems are designed to support communications with several host systems simultaneously. They can have multiple ports and fan-in communications sessions from multiple host HBAs over a single switch link.

A SAN virtualization system can be a dedicated computer system with specialized software for providing virtualization functions, or it can be a networking product that has integrated virtualization capabilities. For instance, a SAN switch or router product can provide virtualization functionality integrated along with its networking services. A dedicated virtualization system can be implemented as an add-in application on a line card and installed in SAN switches.

In-Band Virtualization

Virtualization was initially developed by SAN vendors using two competing architecturesin-band and out-of-band virtualization.In-band virtualization , described in the preceding section, is the most widely implemented architecture and appears to be on its way to being the de facto standard in the industry. In-band virtualization solutions are also referred to as in-path because they operate in the I/O path between systems and storage. As described, they interpret SCSI I/O commands from host initiators and then reformulate them for transmission to downstream storage.

One potential shortcoming of in-band virtualization is the potential for increased I/O latency in SAN operations. It only makes sense that adding multiple initiator/LU sessions would result in transmission delays of some sort. As it turns out, the performance impact might not be an issue in many cases. In fact, I/O performance can even be improved through the use of cache memory in the virtualization system. Performance with virtualization is discussed in the section titled "Performance of SAN Virtualization."

The other potential shortcoming of in-band virtualization is the increased likelihood that transmission errors will occur. Again, it only makes sense that if more connections are being used, the likelihood of having a problem with one of them is greater. For that reason, SAN virtualization systems are sometimes designed with error recovery and redundancy technologies such as hot spare and clustering.

The primary advantage of in-band virtualization over out-of-band virtualization is the lack of host software needed for in-band virtualization. As virtualization software processes run in kernel space, the possibility exists for out-of-band virtualization products to cause severe system problems.

Out-of-Band Virtualization

The out-of-band virtualization architecture is more like a distributed volume management system than the in-band virtualization systems just described. The idea is to control the operations of virtualization software agents running on multiple systems from a centralized management console. Each virtualization agent acts like a volume manager for the system it runs on, but all of them can be working with common, shared storage resources.

The architectural advantage of out-of-band virtualization is the lack of additional latency and complexity in the I/O path. However, its primary disadvantage is the presence of virtualization agents running in the system kernel space. Problems with kernel space software can cause system failures and can be very difficult to troubleshoot.