2.7 Storage Management

---

The need for storage management tools stems from the explosive growth of corporate computer networks in recent years, which not only generate more information than ever before but encourage its use over longer periods of time, requiring ever-larger storage capacity. Databases among large companies have surpassed the terabyte range and are now well into the petabyte range. This growth means that IT managers must be extra vigilant on protecting databases and ensuring their availability. Companies that have kept their storage practices up to pace with the growth of their networks have integrated storage management procedures into network management schemes and are giving due consideration to new distribution and protection methods, including hierarchical storage management, storage area networks, and outsourcing arrangements offered by storage service providers.

2.7.1 Monitoring Disk Use

To remain productive, network users must have an adequate amount of disk space for storing programs and data. To help the LAN administrator meet user demands, a disk monitor agent is used to report volume capacity and manage disk space use. Having current disk use information helps the LAN administrator make decisions about reallocating resources or purchasing additional drives or servers.

The disk monitor agent scans the server volumes to collect disk use information. The collected information includes a list of volumes that were scanned, total volume capacity, and space used. In addition, the time and date disk data was collected is recorded for comparing disk use over a period of time. Some vendors’ disk monitoring tools also record disk usage data based on directory and file owner.

Depending on the product selected, one or more of the following categories of information can be collected by a disk monitor agent and displayed at the console:

• Volumes: Date and time data was collected; server name; volumes scanned, capacity; and total space used and available.

• Directories: Date and time data was collected; server volume and directory names; creation date and time; file count; directory size (in bytes); owner name; and groups to which the owner is a member.

• Directory and file owners: Date and time data was collected; server and volume names; owner name; groups to which the owner is a member; total number of files; and total space used (in bytes).

A disk monitor agent provides automatic notification if disk usage reaches a specified level. Notification can be sent to the person responsible for responding to the situation through a console, network, pager, or e-mail message. The console provides view formats and types that allow the LAN administrator to analyze current disk use as well as disk use patterns over a period of time.

Usually, the disk monitor agent can process data in the background while other programs are running. It also can be made to run interactively, with progress displayed in a status window of the console as volumes are being scanned. The scheduler agent, previously discussed, can be tasked to run the disk monitor automatically in batch mode to collect data at night. This feature is particularly helpful because collecting data from some volumes can take a significant amount of time. Using a scheduler agent, the LAN administrator can regularly collect data each night, once a week, or once a month, depending on organizational requirements. This process ensures that information is collected consistently.

2.7.2 Hierarchical Storage Management

The LAN environment, especially those based on UNIX, are not very friendly when it comes to managing data on disks and tapes. In the UNIX environment, when disk space is exhausted or a job needs an archived file, work stops. Local backups are often so time-consuming that operators often do not bother with them, while remote backups tend to clog the network. Furthermore, tapes are not protected against overwrites, which can result in the loss of important data.

A variety of tools are now available to properly manage storage in the UNIX environment, ensuring that files are backed up at the right time and that tapes are properly labeled and write-protected to prevent data loss. On-line, backup, and archive versions of UNIX files can now be tracked, regardless of the medium on which they reside.

Some tools track file residency, providing for the seamless movement of files from backup to archive and back to the system when needed. Through the common file catalog, the archive facility can locate and initiate a restore of an archived file without the involvement of the LAN administrator. Any user request, process, or program attempting to access an archived file is suspended until the file is restored and then allowed to continue without failure.

With the right storage management tools, users are assured of having enough disk space to accommodate new files. When a file system reaches a predefined threshold of X percent full, automated procedures are initiated that determine which files are eligible for archive and are currently backed up. The file catalog is then simply updated to indicate that the files have been archived and deletes them from the disk file system, freeing up needed disk space.

Files backed up or archived to tape can be assigned an expiration date. Each tape has an internal label that is validated each time it is mounted. File location information, stored in a common file catalog, is compared with the tape label to determine whether the tape may be overwritten. If so, it is considered expired and may be removed from the tape backup or archive pool for reuse. Tapes removed in error cannot be overwritten, and new retention criteria are established for expired tapes when they are reused.

In the movement of files for backup and archival storage, today’s management tools overcome many of the limitations of the UNIX tar program. For example, they offer file compression of 30% to 70% and an encryption scheme to prevent loss of data resulting from tampering. And whereas tar is limited to writing data in 5K blocks, today’s tools optimize performance based on media type and available memory.

2.7.3 Storage Area Networks

With the rapidly increasing volume of mission-critical information being accumulated in today’s business environment, companies are demanding better performance, availability, manageability, and security of their data storage assets. To meet these needs, many companies are implementing SANs. Essentially, a SAN is a specialized network that enables fast, reliable access among servers and external or independent storage resources, regardless of physical location. These resources may be linked via Fibre Channel, Gigabit Ethernet, and emerging Infiniband technologies.

By decoupling storage from computers, workstations, and servers and taking storage traffic off the operations network, organizations gain a high-performance storage network and improve the performance of the LAN. These features reduce network downtime and productivity losses while extending current storage resources. In effect, the SAN does in a network environment what traditionally has been done in a back-end input/ output (I/O) environment between a server and its own storage subsystem. The result is high speed, high availability, and high reliability.

The move to SANs provides organizations with a new level of scalability, and new tools have become available for centralized administration, allowing a much greater degree of flexibility than the traditional network-attached storage paradigm. Because SANs entail an entirely different way of implementing and managing storage, however, it is a good idea to begin with a simple installation, where the need is immediate, and expand its use throughout the rest of the organization as needs justify.

SANs need not be local or regional. Over the wide area, IP can be used in situations where cost containment is a vital concern. For maximum scalability, however, fiber-based ATM networks can be used to extend the range of SANs to any corporate location across the WAN without jeopardizing the integrity of remote data replication or database image transfer operations. If an organization runs multiple types of traffic over the WAN, then ATM is the clear choice. For example, if the organization has separate data networks for IP, SNA, and frame relay—and then wants to add a SAN—all of this traffic can be combined over ATM. With ATM’s quality-of-service and priority-queuing features, there would be no service degradation among the different traffic types.

2.7.4 Outsourcing Storage Management

For most organizations, data storage and management are not core competencies. Emerging Internet-related businesses are especially challenged, given their skyrocketing data storage requirements, combined with limited capital and human resources. Not only is the procurement of hardware, bandwidth, and management tools costly, but even when experienced IT professionals with storage and networking expertise are found, these things are increasingly expensive and difficult to retain. For these and other reasons, there is a growing movement to outsource storage and associated management requirements to a new class of supplier called the storage service provider (SSP).

An SSP sells storage capacity as a pay-per-use utility. The SSP offers companies the means to store and access data without the expense of buying and maintaining hardware and software. The SSP also performs crucial functions such as easy expansion, data backup and restoration, security, disaster recovery, and capacity planning. The monthly cost of going with an SSP is ostensibly more attractive than establishing an in-house storage infrastructure, which can entail hundreds of thousands of dollars to establish a multisite installation, plus the ongoing cost of staff to manage the operation on a 24/7 basis.

SSPs leverage SAN architectures to establish enterprise-wide networked storage. This widely accepted approach allows data to be delivered across the enterprise from a remote location that is managed by an SSP. Users can then implement new technologies as needed, as opposed to limited upgrades because of distributed architectural and management complexity.

SSPs also leverage connectivity capabilities resulting from Fibre Channel and new Infiniband protocols. These architectures currently allow gigabit-per-second transmission rates across metro-area distances, allowing “local” storage performance levels from remotely administered facilities. With SSPs operating their own WANs, storage facilities are linked up to enable seamless access to enterprise data from any corporate location.

SSPs address a variety of basic enterprise storage needs, starting with advising customers on enterprise storage strategies. Most SSPs have a consulting capability or professional services unit that performs needs assessment and recommendations for on-campus storage needs, as well as remote facility implementations. In addition to providing storage space, SSPs also provide customers with backup, disaster recovery, vaulting, migration, and hierarchical storage management offerings. For organizations with very high up-time requirements or multisite demand for data, SSPs provide a cost-effective alternative for heterogeneous data sharing, data replication, multisite data consolidation, and synchronization.

The use of SSPs can also offer cost savings in storage itself. It is estimated that the average cost to manage primary disk storage in-house is $140 per gigabyte per month. In contrast, SSPs offer SAN-based disk storage for rates starting at $40 per gigabyte per month. Also, SSPs allow users to procure storage without large upfront capital costs.

SSPs allow companies to scale their storage needs on demand, to virtually limitless-levels. This offers a solution for companies that find it difficult to obtain storage and hire new personnel as quickly as their storage needs expand. Industry surveys of IT professionals indicate that as much as 85% of companies are currently experiencing a problem with finding qualified storage staff. Outsourcing storage requirements to an SSP solves this problem.

Many SSPs have developed monitoring software to provide customers with several-key capabilities, including the ability to monitor their storage environment around the clock without a large dedicated staff to keep an eye on various performance metrics. SSPs provide comprehensive reporting and analysis of the entire storage environment. Of course, the availability of such tools means that companies do not have to buy their own monitoring tools and hardware platforms, network them together, and hire experienced staff to use them and keep the whole infrastructure running smoothly.

---