Recovery Facility Strategy

Destruction of the centralized IT facility in a disaster may take an extended period to repair or rebuild. In the interim, it may be necessary to restore computer and network services at an alternate site. There are a number of strategies for this recovery facility; each has its own tradeoffs. More than any other decision, the recovery facility strategy determines how soon services will be available again following a disaster.

Some generalizations are worth considering:

1. The tasks required to build a recovery facility are about the same whether they are done before or after the disaster.

2. The only way to decrease the time it takes to get the recovery facility in service is to put some or all of the recovery facility in place before the disaster. The more you put in place before the disaster, the sooner you have IT services after a disaster.

3. The more you put in place before the disaster, the higher your overhead costs to maintain the recovery facilities until the disaster.

4. A disaster is not a certainty . The money spent before the disaster amounts to insurance against lost IT services.

The various recovery facility strategies have developed to address an organization's willingness to gamble with the uncertainty of a disaster.

No Recovery Site

All organizations start out at this point. It allows the organization to include the extent of the disaster in its planning decisions. A minor disaster, such as a leaking roof, may involve rebuilding part of the existing facility. On the other hand, a major disaster like a landslide wiping out a primary facility might call for the creation of a completely new replacement facility at another site.

Temporary recovery facilities may be required to provide critical IT services, such as phones. A minor disaster may mean temporarily relocating equipment in the existing facility, while a major disaster may require a whole new site for the recovery facilities while the replacement primary facilities are being rebuilt.

Two main factors largely determine the time that it takes to reestablish IT services:

• The extent of the disaster

• The quality of the documentation not destroyed by the disaster

Still, maintaining no recovery site is appropriate where IT services are small or restoration of IT services is not critical.

Recovery Partnerships

Some organizations team up with others in a partnership with reciprocal agreements to aid each other in the event of a disaster. These agreements can cover simple manpower-sharing all the way up to full use of a computer facility. Often, however, since the assisting partner has to continue its day-to-day operations on its systems, the agreements are limited to providing access for a few key, critical applications that the disabled partner must run to stay afloat while its facilities are restored. The primary drawback to these kinds of partnerships is that it takes continual vigilance on behalf of both parties to communicate the inevitable changes that occur in computer and network systems so that the critical applications can make the necessary up-front changes to remain operational. Learning that you cannot run a payroll, for instance, at your partner's facility because it no longer uses the same computer hardware or operating system is not acceptable.

One of the most critical issues involved in the recovery process is the availability of qualified staff to oversee and carry out the tasks involved. This is often where disaster partnerships can have their greatest benefit. Through cooperative agreement, if one partner loses key personnel in the disaster, the other partner can provide skilled workers to carry out recovery and restoration tasks until the disabled partner can hire replacements for its staff. Of course, to be completely fair to all parties involved, the disabled partner should fully compensate the assisting partners for use of its workers unless there has been prior agreement not to do so.

The use of reciprocal recovery agreements of this nature may work well as a low-cost alternative to hiring a disaster recovery company or building a hot recovery site. In addition, reciprocal recovery agreements can be used in conjunction with other arrangements, such as the use of a cold recovery site, described below. The primary drawback to these agreements is that they usually have no provision for providing computer and network access for anything other than predefined critical applications. Therefore, users will be without facilities for a period until systems can be returned to operation.

The same two factors still largely determine the time it takes to reestablish IT services:

• The extent of the disaster

• The quality of the documentation not destroyed by the disaster

Cold Recovery Site

A cold recovery site is an area physically separate from the primary facility where space has been identified for use as the temporary home for the computer and network systems while the primary facility is being repaired. There are varying degrees of "coldness," ranging from an unfinished basement to space where the necessary raised flooring, electrical hookups, and cooling capacity have already been installed, just waiting for the computers to arrive .

The following factors determine the time it takes to reestablish IT services:

• The degree of "coldness" in the cold recovery site

• The quality of the documentation not destroyed by the disaster

Contracted Disaster Recovery Company

A number of companies provide disaster recovery services on a subscription basis. For an annual fee (usually quite steep), you have the right to a variety of computer and other recovery services on extremely short notice in the event of a disaster. These services may reside at a centralized hot site or sites that the company operates, but it is necessary for you to pack up your backup tapes and physically relocate personnel to restore operations at the company's site. Some companies have mobile services, which move the equipment to your site in specially prepared vans. These vans usually contain the entire necessary computer and networking gear already installed, with motor generators for power, ready to go into service almost immediately after arrival at your site. (Note: Most disaster recovery companies that provide these types of subscription services contractually obligate themselves to their customers to not provide the services to any organization who has not subscribed, so looking to one of these companies for assistance after a disaster strikes will likely be a waste of time.)

The following factors determine the time it takes to reestablish IT services:

• The extent of the disaster

• The clarity of the contractual obligation

• The ability of the contracted vendor to deliver as obligated when a disaster occurs

• The quality of the documentation not destroyed by the disaster

Hot Site

This is probably the most expensive option for being prepared for a disaster and is typically most appropriate for very large organizations. A separate computer facility, possibly even located in a different city, can be built, complete with computers and other facilities ready to cut in on a moment's notice in the event that the primary facility goes offline. High-speed communications lines must join the two facilities so that users at the primary site can continue to access the computers from their offices. Hot sites, however, do not have to be interlinked with the primary facility. A secondary live facility, with computer equipment of similar size, operating system, and so on can be considered a hot site in that the only requirement to become operational is data and application restoration.

The following factors determine the time it takes to reestablish IT services:

• The extent of the disaster

• The degree of "heat" in the hot site

• The quality of the documentation not destroyed by the disaster.