Chapter 14: Backup and Recovery

Because of the importance of the Exchange databases and transaction logs, backing up those components is essential. There are also a number of other components that you will want to include when backing up an Exchange server. When Exchange Server 2003 is installed, an Exchange-aware version of Windows Backup replaces the existing version that comes with Windows Server 2003. This chapter begins with an overview of backup technologies and strategies and then looks at using Windows Backup to back up and restore an Exchange server.

Understanding Backups

Before you fire up your backup program and start backing things up, it ‚ s important to have a clear understanding of the technologies involved and to create a good backup plan. This section looks at the various components of Exchange and Windows that you should back up, the types of backups available to you, and several backup strategies.

What ‚ s New with Windows Backup

The Windows Backup utility in Windows Server 2003 has undergone two dramatic improvements since its last release in Windows 2000 Server. The Emergency Repair Disk (ERD) has been replaced by the new (and much improved over ERD) Automatic System Recovery (ASR) functionality. In addition, Windows Server 2003 includes Volume Shadow Copy (VSC) support.

VSC is implemented in two ways in Windows Server 2003. When used in Windows Backup it allows a backup process to create an instant and exact copy of the data to be backed up at the time the backup is initiated. This snapshot is then written to the backup media instead of the original files being referenced while writing to the backup media. When VSC is used in this way, it allows a safe and effective mechanism to back up files that are open at the time of the backup. These files, which would normally be skipped , are backed up in a closed state as of the time the snapshot is taken and thus appear closed on the backup media. Running applications are not affected by this process and continue to run unaffected by the backup. In order to use VSC, the volume must be using NTFS (New Technology Filesystem). VSC can also be disabled during the creation of a backup job if desired.

What to Back Up

An Exchange server is composed of a great deal of information, including the Exchange databases of user messages and public messages and the transaction logs associated with those databases. Configuration information is stored in the Microsoft Windows Registry, in various places in the Exchange Server installation path , in Active Directory, and even on some users ‚ computers. This section covers the information that you should include when backing up an Exchange server.

Note ‚

Much of the information in this section is a recap of how the Information Stores in Exchange Server 2003 work. You can learn more about the Exchange storage architecture in Chapter 2, ‚“Microsoft Exchange Architecture. ‚½

Databases

Much of the information in Exchange Server 2003, including private and public user messages, is stored in two databases: PRIV x . EDB and PUB x .EDB.

PRIV x .EDB Each mailbox store database on an Exchange server is named using the format PRIV x .EDB , where x ranges from 1 to the number of databases on the server. The private store databases hold user mailboxes and messages. By default, these databases are located in Program Files\Exchsrvr\Mdbdata\ .

PUB x .EDB Each public store database on an Exchange Server is named using the format PUB x .EDB , where x ranges from 1 to the number of databases on the server. The public store databases hold messages and documents stored in public folders. By default, these databases are also located in Program Files\Exchsrvr\Mdbdata\ .

In addition to these core databases, a streaming database with the extension .STM is associated with each mailbox and public database. Several optional databases might also be available on any given Exchange server. These optional databases represent various services that may be installed on a server, such as a Site Replication Services (SRS) database .

Transaction Logs

Whenever a transaction occurs on an Exchange server, that transaction is first recorded in a transaction log . Transactions are written to the database later during idle time. Transaction logs are the primary storage areas for new transactions. One set of transaction logs exists for each storage group on a server. A set is composed of a current log, any number of previous logs, reserve logs, and a checkpoint file. The current transaction log file, named EDB.LOG, resides in the \Exchsrvr\Mdbdata directory by default.

Data is written to log files sequentially as transactions occur. Regular database maintenance routines commit changes in the logs to the actual databases later. The most current state of an Exchange service, therefore, is the .EDB database and .STM database, plus the current log files (including the checkpoint file) that have not yet been committed to the database. Thus, transaction logs are an essential part of the backup routine.

Checkpoint files are used to keep track of transactions that are committed to the database from a transaction log. Using checkpoint files ensures that transactions cannot be committed to a database more than once. Checkpoint files are named EDB.CHK and reside in the same directories as their log files and databases.

Log files are always given 5 MB of reserved disk space. When a log file fills, it is renamed, and a new log file is created. Old, renamed log files are called previous logs . Previous logs are named sequentially, using the format EDB xxxxx .LOG, in which each x represents a hexadecimal number. Previous logs are stored in the same directories as their current- log-file counterparts.

During an online backup of an Exchange server, previous log files that are fully committed are purged. Previous log files can still consume a good deal of disk space. Exchange Server 2003 provides a feature called circular logging that can help prevent that waste of disk space. When circular logging is enabled, only previous log files with uncommitted changes are maintained for each storage group. This can significantly reduce the amount of hard disk space that is required for your Exchange server, compared with keeping all transaction logs until a backup is completed. Circular logging is disabled by default, but you can enable it on the General property sheet of a storage group container in System Manager.

Note ‚

You can find more information on using circular logging in Chapter 9, ‚“Configuring the Information Store. ‚½

In addition to all of the current and previous transaction logs, the online backup process also creates patch files that serve as temporary logs to store transactions while the backup is taking place. Transactions in these logs are committed when the backup is finished.

Other Items to Back Up

In addition to Exchange databases and transaction log files (all of which are included automatically in a regular online backup of Exchange), you will want to consider several other items:

EXCHSRVR subdirectories Many valuable pieces of information, including message-tracking data, are located in various subfolders of Program Files\Exchsrvr .

Site Replication Service (SRS) database The SRS database is used in mixed Exchange Server 5.5 and Exchange 2000 Server/Exchange Server 2003 environments. You can learn more about the SRS database itself in Chapter 11, ‚“Coexisting with and Migrating from Exchange 5.5. ‚½

User information Many administrators allow the storage of users ‚ personal folders (PST files) and address books on the Exchange server or another network server. Always make sure that information of this sort is included in your backup strategy. When users store personal folders on their local workstations, you will need to involve the users in the backup procedure.

Backing Up System State Information

A System State backup is used to back up configuration information critical to a Windows Server 2003 computer. You ‚ ll learn how to create a System State backup later in the chapter. System State information includes the following:

Windows 2003 Registry The Windows Registry contains a great deal of configuration information relating to Exchange Server 2003, especially information relating to the coexistence of Exchange Server 2003 with previous versions. You should perform regular System State backups on Exchange servers to include the Windows Registry.

Internet Information Services metabase Since Exchange Server 2003 relies so heavily on IIS for protocol support, it is only natural that IIS be a part of any good backup plan. The IIS metabase, a database of configuration information, is included in a System State backup. Running regular System State backups on the Exchange server also backs up the IIS metabase.

Active Directory While Active Directory is not really a part of Exchange, most of the configuration information for Exchange Server 2003 is stored in Active Directory. Recipient objects are also kept in Active Directory. You should run regular System State backups of domain controllers to capture Active Directory information. You should back up Active Directory and Exchange Server 2003 databases at the same time to avoid losing configuration of user objects on the domain controllers or in global catalogs.

Preparing a System for Backup and Disaster Recovery

When your Exchange server has only one hard disk, system information, databases, and log files are all stored on that disk. More often, however, an Exchange server is configured with more than one local hard disk. This is because Exchange offers flexibility in partitioning data across multiple locations to help with performance and for purposes of restoration.

In general, we recommend the following practices to optimize performance and disaster recovery:

Keep log files on separate hard disks. The speed of an Exchange Server 2003 database depends a good deal on how quickly transactions are copied from memory to the transaction log. For this reason, we recommend that you place the log files for each storage group on a dedicated hard disk so that the transaction logs do not compete with any other read/write operations. The best performance is usually obtained from a mirrored disk device. Also, it is a good idea to keep transaction logs on a separate disk from the Information Store to ensure recoverability.

Keep storage groups on separate disks. We also like to keep storage groups on separate disks from other storage groups. This decreases the damage that a single drive failure can do. However, it is more important to have the transaction logs on a separate disk than to separate the storage groups themselves . Storing log files on separate disks dramatically increases your odds of successful recovery in the event of failure. For example, if you had three free hard disks (aside from the hard disk holding system information) and two storage groups, it would be better to put the two storage groups on one disk and then to put the transaction logs for each storage group onto their own disks than to try to separate the storage groups. In Figure 14.1, storage groups 1 and 2 are placed on a single disk array (F:). The logs for each storage group are placed on their own volumes (D: and E:).

Figure 14.1: Separating log files is more important than separating storage groups.

Keep databases small. Within a storage group, data can be partitioned into multiple databases. We recommend that you keep these databases smaller rather than larger. If you have a particularly large database, dividing it into multiple databases improves recovery for two reasons:

Users can begin accessing a database as soon as it is restored.
If you use multiple backup media, you can actually restore the multiple databases in parallel. Restoring a smaller database takes less time than restoring a larger database.

Turn off circular logging for all storage groups. The disadvantage of using circular logging is that it prevents you from using differential or incremental backups (discussed later in this chapter). Also, because some log files are discarded before backup, you may not be able to fully restore a server by replaying the log files, a potentially serious situation. For this reason, we generally recommend that you leave circular logging disabled for all your Exchange servers.

Note ‚

You may want to turn on circular logging when importing a large amount of data. However, you must remember to disable it afterward and do a full backup. You may also want to turn on circular logging for servers where recovery is not so important, such as for front-end servers that do not contain mailboxes or public folders.

Back up whole storage groups at once. We recommended that you back up an entire storage group at the same time. This makes the backup process easier to manage because the databases within the storage group share the same set of log files. The log files can be truncated only after all databases have been backed up.

Back up storage groups to different media. To prepare for a failure of all storage related to a server, you could partition data into separate storage groups and back up these storage groups to different media. As long as you initiate separate restore sessions for each storage group, Exchange can restore multiple storage groups at the same time. The storage groups should also be hosted on separate physical disks and RAID disk arrays so the physical disk media do not become a bottleneck for the restore. This is known as a parallel restore.

Backup Methods

Exchange Server 2003 supports two methods of performing backups on your databases: online and offline. When possible, you should always strive to perform online backups because service to your users is not disrupted during the backup when databases are left online. Users will continue to have access to mailbox and public folder stores with little to no difference in performance.

Online backups An online backup, as discussed below in Exercise 14.1, backs up the EDB, STM, and LOG files that constitute the database. Each of these files is checked for corruption at the file level during the backup process by verifying that the checksums on each 4-KB page in the database are correct. Should a checksum failure occur, the online backup will fail and you will receive a report from both the Windows Backup utility and one or more Application Log entries related to the event, indicating that the backup could not completed. The failure of a checksum is indicative of larger database corruption issues.

Offline backups An offline backup, as you might expect, requires that you first dismount the database to be backed up, thus making it unavailable for user access. Should online backups fail because of checksum errors, an offline backup may be performed. You may also find yourself needing to do offline backups if you are using a third-party backup application that does not support the Exchange online backup APIs (Application Programming Interfaces).

Besides taking the database out of use, offline backups do not purge committed transaction logs after the backup has completed. In addition, no automatic checksum error checking is performed during an offline backup. You will need to use the eesutil utility to check the offline backup for corruption after it has been completed.

Types of Backups

You can perform five basic types of backups using the Windows Backup utility (and most other backup programs). The key difference between these backup types is how each one handles the archive bit in every file. When a file is created or modified, the archive bit is set to on. When some types of backups run, the archive bit is set to off, which indicates that the file has been backed up.

The five backup types are as follows :

Normal During a normal backup , all selected files are backed up, regardless of how their archive bit is set. After the backup, the archive bit is set to off for all files, indicating that those files have been backed up. Upon the completion of the normal backup, any transaction logs that have already been committed to the Exchange database are purged from the server.

Copy During a copy backup , all selected files are backed up, regardless of how their archive bit is set. After the backup, the archive bit is not changed in any file. Copy backups do not purge committed transaction logs from the server.

Incremental During an incremental backup , all files for which the archive bit is on are backed up. After the backup, the archive bit is set to off for all files that were backed up. Upon the completion of the incremental backup, any transaction logs that have already been committed to the Exchange database are purged from the server.

Differential During a differential backup , all files for which the archive bit is on are backed up. After the backup, the archive bit is not changed in any file. Differential backups do not purge committed transaction logs from the server.

Daily During a daily backup , all files that changed on the day of the backup are backed up, and the archive bit is not changed in any file. Daily backups are not a sensible Exchange server backup method and thus are not recommended or discussed any further.

Backup Strategies

Although there are many backup strategies, three basic strategies serve most purposes. Table 14.1 describes these three basic strategies, along with some of their advantages and disadvantages. A five-day workweek is assumed.

Table 14.1: Three Basic Backup Strategies
Backup Strategy	Description	Advantages	Disadvantages
Full daily (also called normal)	A full (i.e., complete) backup is performed every day. Given the storage capacity and speed of modern backup devices and that the Windows Backup utility allows you to back up to any available drive, daily full backups are the choice of most Exchange administrators, and we recommend them.	Only one tape is needed to perform a restoration. This strategy requires the least amount of time to restore ( assuming an entire backup fits onto one tape).	This strategy requires the longest amount of time to perform the backup.
A full backup once per week and an incremental backup every other day	A full backup is done on day one. An incremental backup (using a new tape every day) is performed every day for the next four days. This procedure backs up only the new and changed data since the last full or incremental backup (whichever is more recent).	This strategy takes the least amount of time to back up.	This strategy could require up to five tapes to perform a restoration. This strategy takes the most time to restore.
A full backup once per week and a differential backup every other day	A full backup is done on day one. A differential backup (using a single new tape) is performed every day for the next four days. This procedure backs up only the new and changed data since the last full backup.	No more than two tapes (the full and the last differential) are required to perform a restoration, making restoration faster than with the incremental method.	This strategy takes progressively longer to back up each day.

Restoration Strategies

Having a backup plan is only one part of a disaster recovery plan. You must also plan for restoration (among other items not discussed here, such as media storage and rotation patterns). There are three basic means you can employ to restore an Exchange database: restore in place directly to the storage group, restore using an alternate recovery forest, or restore using the new recovery storage group . Exercise 14.2 illustrates the very straightforward restoration directly to the affected storage group; this method is not discussed any further here.

As mentioned previously, the recovery storage group is a new feature in Exchange Server 2003 that provides an easy-to-use storage group for the restoration of databases and the recovery of mailboxes only ‚ you cannot use it to hold production databases. In order to use the recovery storage group to recover a mailbox store, you must keep the following limitations in mind:

Public folder stores cannot be restored using the recovery storage group.
The storage group was backed up from an Exchange Server 2003 computer or an Exchange 2000 Server SP3 (or later) server.
You can restore only one storage group at a time, but this storage group can contain multiple mailbox stores.
The Exchange server holding the recovery storage group must be located in the same administrative group as the server from which the storage group was backed up.
Only one backup set can be used to perform the restoration.
Adequate disk space must exist on the recovery storage group server to hold the temporary data.
Volume Shadow Copy backups cannot be restored using the recovery storage group method.
The Exchange server hosting the recovery storage group must be running exactly the same version of Exchange as the server from which the storage group was backed up. This requirement is due to the different log file formats that exist between Exchange 2000 Server and Exchange Server 2003.
When using the recovery storage group to recover a single mailbox, the mailbox owner must still be in Active Directory and must have a mailbox.

Creating the Disaster Recovery Plan

Determining your backup methodology and your restoration methodology is just the beginning of a complete disaster recovery plan. Several other important issues must be considered. The following are a partial list of these items, but the list should not be considered full and complete by any means:

What sort of media rotation scheme will be used? Using the same backup media each day will quickly deteriorate the physical condition of the backup media. A well-planned media rotation scheme can provide a backup history of weeks, months, or even years , depending on the scheme used.
Where will the backup media be stored? Storing your backup tapes in the file cabinet in your office does little good if your building is flooded or catches fire. Several companies exist today that will pick up and store your backup media in a secure location. They also offer the ability to have backup media returned to you in the event you need it for restoration. Note that these companies can also help with a media rotation scheme.
Who will be responsible for maintaining, testing, and validating the disaster recovery plan? If no one is responsible for making sure the plan works, then you ‚ ll likely have an unpleasant surprise when it comes time to perform a restoration for real.
Who will be part of the disaster recovery team? Depending on the size of your organization, you may need to group several employees into a disaster recovery team. It may also be useful to include representatives from outside the IT department. These are the people who will want to know when their data is coming back.
Realize that Exchange is not the only critical item that needs to be backed up. While e-mail is more mission-critical now than it has ever been in the past, your company likely has many other vital pieces of data that need to be treated with the same concern as the Exchange databases. SQL databases, file shares, and Active Directory data all have a direct impact on your company ‚ s ability to function properly.

So far, it may seem like using the recovery storage group is not a very useful option ‚ nothing could be further from the truth. When you use the recovery storage group to restore a database, you achieve significantly reduced mailbox store downtime. When you use the recovery storage group method, the store must be dismounted only when the data is moved into place.

The last restoration method, the alternate recovery forest, is the most complex restoration method, but it has some significant advantages, which we ‚ ll examine momentarily. To perform restorations using this method, you must first create another Active Directory forest that supports an Exchange server installation. This forest must have the exact same names as the production forest you ‚ re restoring from. In most cases, you can use a single server to create the forest and host the Exchange installation, thereby reducing hardware and software expenses. This server, however, does become a dedicated restoration server and cannot typically be used for any other network purpose. The alternate recovery forest provides the following benefits:

Restorations can be performed and tested in a nonproduction environment.
Public folder stores can be restored.
Backups made using Volume Shadow Copies can be restored.
Purged mailboxes can be removed after the mailbox-deletion retention period has passed.

As mentioned previously, the only real drawbacks to this method are the complexities involved with creating and maintaining this environment and the hardware and software investment that must be made.