11.2 In-Band and Out-of-Band Virtualization

In traditional shared storage access, the path between the server and the networked storage targets carries both the information about how data is stored (metadata) and the data itself, as shown in Figure 11-4. A file system, for example, may reside on a physical storage array, containing metadata such as file names, properties, and lists of block addresses where each file's data is stored. The rest of the array is composed of the data blocks to which the file system metadata points. The host system thus accesses metadata and the data itself in-band, through the transport path established by the SAN between server and target array.

Figure 11-4. Conventional in-band access to metadata and data over a SAN

What happens, then, when a virtualization engine abstracts both the view of metadata and the physical location of data blocks? If the virtualization engine remains in the path between servers and storage, it preserves in-band access. Alternatively, the virtualization engine can split the access path to metadata from the access path to data, creating an out-of-band implementation. As shown in Figure 11-5, the server's view of metadata shifts from physical storage to the virtualization entity, which in turn controls how data blocks are physically distributed in the storage pool. In reality, the metadata always sits somewhere on physical disk media, whether in the storage pool or on dedicated disks within the virtualization product.

Figure 11-5. In-band and out-of-band virtualization solutions

Vendors of each solution are quick to point out the superior benefits of in-band or out-of-band virtualization methods. The in-band architecture is the least intrusive from the standpoint of the server. Instead of discovering and attaching to multiple storage targets across the SAN, the server now sees only a single large storage resource in its path, represented by the virtualization appliance. The in-band virtualization engine manages the disparate physical storage devices in the downstream SAN and presents a coherent image of metadata to the server. But because the virtualization engine in this case presumes to sit directly between servers and storage, it is a potential bottle neck as more servers and storage are added. Vendors must therefore demonstrate that their in-band solutions do not impede data throughput and can handle both virtualization processing and data transport at wire speed. Performance can be enhanced by incorporating the virtualization engine directly into the SAN interconnection, melding switching and virtualization functions into a single product. This both streamlines the virtualization solution and leverages the ability of a SAN to support heterogeneous storage devices.

Whether appliance-based or switch-based, in-band virtualization may be vulnerable to disruption. Because the in-band appliance now becomes the sole gateway between servers and storage, failure of the appliance can result in loss of data access for the entire SAN. A distributed virtualization solution could resolve this problem by allowing multiple appliances to share metadata information and provide failover in the event of disruption.

The out-of-band architecture avoids in-band issues by placing metadata control outside the data transport path. Individual servers, however, must have virtualization software agents installed so that I/O requests can be redirected to the out-of-band appliance. Separating data and control paths ensures that the virtualization appliance will not act as a bottleneck to data transport, although, as with in-band methods, loss of the out-of-band appliance itself can still be disruptive. Redundant out-of-band appliances with synchronized I/O control could be configured to avoid outages and provide high availability.

The debate between the in-band and out-of-band approaches is tied to another virtualization quandary: Where should the virtualization intelligence reside? The three commonly proposed sites are in the host, in the interconnection, and in the storage target. Each location has its strengths and weaknesses, and some approaches attempt to distribute virtualization support across all three.