10.1 Storage Network Management

As long as the server/storage relationship was isolated from the network, storage management applications tended to be focused on the server (the direct owner of storage) and specific operating systems. Storage networking shifts the focus from the server to storage and enables storage management to address data issues apart from individual servers and independently of the operating platform. With SANs, NT and UNIX servers can now share storage resources of a single disk array, and management of the array's resources may exist independently for example, as a storage management entity of either an NT or a UNIX environment. This abstraction from the previous server-centric model promises far more flexibility for resource allocation, volume management, and data integrity.

As shown in Figure 10-1, management of the SAN is a hierarchy of functions that may exist as separate applications or as integrated management systems. Lower layers move status and event information up through the hierarchy, and upper layers issue commands and queries down to the appropriate agents. The management structure is built on a foundation of managed devices that create the SAN interconnection (host adapters, switches, and bridges) as well as management entities that may reside in disk arrays and tape subsystems. These devices communicate to their respective device management applications via a number of protocols, primarily Simple Network Management Protocol (SNMP), SCSI Enclosure Services, or proprietary APIs. The device management applications, in turn, may communicate to upper-level storage and storage resource managers, which may provide interfaces to enterprise-level systems management platforms. SAN hardware and software vendors have a vested interest in adapting their products to this umbrella management strategy, because cohesiveness of the different strata that compose SAN management will facilitate deployment of all products.

Figure 10-1. SAN management hierarchy

Storage network management is responsible for interfacing directly with interconnection equipment. The vendor of a managed network device supplies a console or graphical interface that, at minimum, allows port configuration (insert/bypass) and reports basic enclosure environmental status (power, fan, temperature). The feature set of a managed HBA, hub, or fabric switch is vendor-dependent. Some products offer only basic enclosure and port status, whereas others include performance and diagnostic utilities.

The application provided by the vendor to manage its product is a device manager, because typically the management scope does not extend beyond the product itself. A management utility for a host adapter card, for example, will give status, configuration, and port statistics for the adapter, but it may not provide visibility to fabric switches or other nodes with which it communicates.

The management of multiple storage network products implies the need for multiple device managers, and consequently multiple management workstations or consoles to support applications from different vendors. Maintaining multiple management consoles is not an attractive option for IT administrators. Paralleling the evolution of network management in local and wide area networking, this fragmentation of management platforms provides incentives for consolidating device managers under one storage network application.

Device management has software, firmware, and hardware components, as shown in Figure 10-2. Putting a port into bypass mode from a management console, for example, requires use of a management protocol (such as SNMP) to convey the command from the workstation to the device. A management agent on-board the device must then translate the bypass command into a hardware instruction, and the hardware, in turn, must raise or lower the appropriate leads to physically bypass the port.

Figure 10-2. Device management architecture

The controller or firmware that supports the management agent is the device's voice to the outside world. Communication between a device and its management workstation may occur through the storage transport media, and this is referred to as in-band management. Alternatively, management data may be segregated from storage data traffic via an external Ethernet, RS-232, or other interface, and this is referred to as out-of-band management. Out-of-band management through Ethernet normally uses the SNMP protocol, although Telnet and Web browser HTTP (Hypertext Transfer Protocol)-based implementations are also available. In-band and out-of-band solutions have their respective advantages and disadvantages, and some products offer both to provide complementary functions.

10.1.1 In-Band Management

As illustrated in Figure 10-3, in-band management relies on the SAN transport to carry status and configuration information between devices and a management console. The management protocol may be SNMP-based or may be a combination of APIs that interface to the management software application. In a Fibre Channel SAN, you might use the common HBA API to report host bus adapter status and use proprietary switch and storage APIs to report status and configuration information for the fabric and targets. In the case of an IP SAN, SNMP or API queries could be sent along the same infrastructure as the SAN interconnection, directed at the IP addresses of management entities within host, switch, or target devices.

Figure 10-3. In-band management over the SAN data path

In-band management simplifies SAN deployment, provided that all the devices that require management support in-band methods. Unfortunately, in Fibre Channel SANs some devices still require out-of-band access, so you need an additional Ethernet network to handle SNMP traffic.

The significant drawback of any in-band management scheme is apparent when the storage transport itself is down or disabled by excessive errors. Because all management data must move across the storage network, loss of the transport means loss of management visibility to the managed devices, and that negates the ability to detect, isolate, and recover from network problems. Thus, although in-band management fulfills a useful function under stable network conditions, it is ill equipped to deal with catastrophic physical-layer or transport-level events. In large, meshed SANs, you can somewhat obviate this vulnerability by providing redundant links. Providing alternative paths for storage data and management data, however, adds cost, consumes additional ports, and still does not eliminate the possibility that a management workstation will be rendered ineffectual if combinations of links or fabrics fail.

10.1.2 Out-of-Band Management

Out-of-band management avoids in-band issues by moving all management data from the production storage network. As shown in Figure 10-4, out-of-band techniques typically use Ethernet for the management data path and wrap management queries or commands in SNMP, Telnet, or Web browser HTTP protocols. Most switches and storage targets also provide a serial RS-232 console port as an alternative (but rarely used) access method.

Figure 10-4. Out-of-band management requires a separate network to carry management data

Because out-of-band implementations do not rely on the SAN transport, you can still manage host adapters, switches, and storage arrays if the fabric links are down. This is a significant advantage over in-band methods. Out-of-band management also facilitates integration of storage network management with enterprise management platforms, particularly those based on IP and SNMP. The major weakness of out-of-band network management in Fibre Channel environments is its inability to provide auto-topology mapping and other functions that require in-band communication between devices.

The most commonly used out-of-band protocols are SNMP, HTTP, and Telnet, all of which run over IP. The use of IP to carry management instructions and responses lets you manage storage network devices from anywhere in a routed IP network. Storage network management can thus be co-located with the SAN or can be centralized with other IP-based network management applications in an enterprise network operating center (NOC).

10.1.3 SNMP

SNMP (Simple Network Management Protocol) is the predominant protocol for multi vendor enterprise networks and is widely supported by routers and switches in wide and local area networking. SNMP provides a command set for soliciting status (SNMP Gets) or setting operational parameters (SNMP Sets) of target devices. An enterprise-wide SNMP management platform is typically run as a graphical interface on a large UNIX or NT workstation and may poll hundreds of devices throughout the routed network. The management platform contains the SNMP manager; the managed router, hub, or switch contains an SNMP agent.

Device status information may include a variety of data points: serial number, vendor ID, enclosure status, port type, port operational state, traffic volumes, error conditions, and so on. This information is organized in a management information base (MIB), which is maintained by both the management workstation and the SNMP agent within the managed device. For LAN and WAN management, several standard MIBs have been sanctioned by the internetworking community through a series of RFCs. If a vendor wishes to include additional device information that is not specified in a standard MIB, vendor-specific parameters or status can be compiled in a proprietary enterprise MIB or MIB extension.

Device information, status, and control variables within an MIB are organized in a hierarchical data structure called structure of management in formation (SMI). SMI defines an information tree whose branches lead to various management information bases and whose leaves are discrete data about a device's functionality and status. SMI notation, which is carried within an SNMP query or command, is essentially an address pointing to the location of the data requested by the management workstation.

In the MIB example given in Figure 10-5, the SMI notation 1.3.6.1.4. 1.2490.1.1 points along multiple branches of the management tree and terminates at a vendor's MIB extension for Fibre Channel hub variables. Below this point, additional branches may lead to specific port, environment, and diagnostic values. With the adoption of standard Fibre Channel MIBs, common features shared by all vendors migrate to standard MIBs, whereas vendor-specific functionality will remain in proprietary MIB extensions. This structure accommodates further feature enhancements without restricting vendors to the lowest common denominator management capability.

Figure 10-5. SNMP SMI notation for a Fibre Channel loop hub device

In addition to providing the ability to actively query a device for status, SNMP allows a device to generate a trap, or unsolicited status information. If a preconfigured error condition or threshold is reached, the managed device will initiate an SNMP message to the management workstation as an alert. On the management platform, the icon representing the managed device typically turns yellow or red, and the application may send a page or e-mail to the network operator. Because SNMP is a protocol shared by the vendor's device management application and third-party management platforms, SNMP traps can be directed to either one. Integration into HP OpenView, for example, would allow a trap to be addressed to the OpenView management workstation, which could then automatically launch the vendor's device manager for further status or diagnosis of the problem.

Any management facility based on IP implies additional traffic on the messaging network. Although polling for all status information from mul tiple SNMP agents may generate noticeable overhead, SNMP management applications can reduce IP traffic by relying on traps for notification of events or by polling only for changes from an original status check.

10.1.4 HTTP

Use of HTTP (Hypertext Transfer Protocol) is a management tool that leverages the proliferation of Web browsers for managing network products. Web-based management of network resources is being extended through the Web Based Enterprise Management (WBEM) initiative, which makes browsers the common interface for managing diverse network components.

An HTTP management implementation has two components: an HTTP server that resides in firmware in the managed device, and a Web browser (such as Netscape or Explorer) that acts as a management graphical interface. By pointing the browser to the IP address of the managed device, the user can navigate a series of screens to monitor device status or change device parameters.

Embedded HTTP capability is an entry-level management scheme and does not integrate into broader management applications as easily as SNMP or SES. Because a browser can be addressed only to one IP target at a time, the administrator would have to open multiple instances of a browser application to manage multiple HTTP-based devices. Embedded HTTP management does not facilitate a comprehensive view of all managed products. Management-by-browser is therefore more suited to small SAN installations with a single fabric to monitor. Recognizing that HTTP-based management has an appeal for the low end of the market, vendors of SAN products may provide both HTTP and SNMP management options.

Even with password protection at the managed device, HTTP presents a higher security risk than other methods. All that is required to access an HTTP-managed device is a browser (which everyone has) and the proper IP address (which can be hacked). At minimum, password protection should be provided for both read-only and read/write access, complemented by a firewall to prohibit external penetration of the customer network.

Browser-based management can also be implemented in a client/server configuration, as shown in Figure 10-6. A management workstation communicates with devices using SNMP and concurrently provides an HTTP server interface to remote browser consoles. This arrangement allows mul tiple remote managers to view the storage network via browsers and overcomes the single-device limitation of embedded HTTP management. By assigning read-only and read/write permissions, the administrator can provide management operations for monitoring and modification controls.

Figure 10-6. Browser-based HTTP client/server management

10.1.5 Telnet

Unlike typical SNMP- or HTTP-based management solutions, a Telnet implementation is a text-only, command line interface. The user sends a Telnet command across the IP network to the managed device (for example, Telnet 192.168.1.1) and establishes a session. The vendor, hopefully, has provided one or two layers of password protection, beyond which a menu of commands is available. Telnet is usually used only to set the device's default IP address to one that can be used for SNMP access, but a command line interface alone is sometimes the tool of choice for UNIX environments.

The command set available and status information that can be queried via Telnet are vendor-dependent. Some offer only basic configuration commands, whereas others provide commands and queries for all functions that are available through the vendor's SNMP graphical interface.

10.1.6 Storage Network Management Issues

Storage network management faces several challenges previously encountered in local and wide area networking. In complex multivendor, multiplatform environments, customer requirements cannot be met with proprietary management solutions, even if a vendor offers a rich feature set for man agement of its own products. Open systems exert pressure on all vendors to seek common methods cooperatively while competing for market share with unique functionality. This contradiction slowly and sometimes painfully works its way through standards bodies and industry groups, with both technical and political consequences. Still, the net result of this process is always preferable to a closed, proprietary approach.

Because storage network management is shaped by the methods and capabilities of individual device managers, the configuration, status, and diagnostic features of each Fibre Channel network product are more useful if they are accessible through a common platform. Development of standard Fibre Channel MIBs, for example, simplifies the creation of comprehensive network management applications and enables high-level storage management platforms to gather data from the SAN interconnect. From the customer's standpoint, the ability to manage a variety of products from a common interface has significant value because it consolidates network management consoles and streamlines staffing and training requirements.

A common management language also facilitates network management of the SAN. A single enterprise may have NT, Solaris, AIX, HP-UX, and other operating systems spread across the network. Network management may be centralized in a NOC or may be dispersed throughout individual business units. To accommodate the variety of platforms on which management applications may be run, the trend in SAN management has been toward a standardized management interface suitable for multivendor SANs and one that can be implemented on a variety of management platforms. As discussed later in this chapter, the Common Information Model (CIM) is being developed to meet these requirements.