Understanding Exchange Backup Methods and Requirements

Now that you have a better idea how the Exchange databases are structured and data flows through the Exchange processes, you will be better able to understand the different backup methods used with Exchange and the requirements to back up the databases successfully. There are basically two types of Exchange backup: online and offline.

Online Backup

Online backups are made while the Exchange 2003 services are running and use the Exchange streaming backup API. The Exchange 2003 online backup API automatically synchronizes and gathers the Exchange 2003 database and transaction log data that will be required for successful restoration using the same channel as normal database access. Although this can take longer than offline backups, it has the advantage of not interrupting users' use of email services, and as each 4K page is passed through the database engine to be written to backup media, its checksum is verified. Any error found is reported as a -1018 error, and the backup is terminated.

Figure 13.8 shows a representation of the Exchange backup process.

Figure 13.8. The Exchange backup process.

There are four types of online backups of an Exchange database: normal, copy, incremental, and differential. In Exchange 2003 these processes are as described in the following sections.

Normal (Full) Backup

The Backup application backs up the ESE database files (.edb and .stm) and at least one of the log files. On completion, the backup application deletes all committed log files indicated by the pointer in the checkpoint file. A patch page is then added to the database containing details of messaging transactions that occurred in Exchange while the backup process was running.

Note that without circular logging, or an Exchange-aware backup process such as this, the log files would accumulate until all disk space was consumed. This is a common problem with badly configured and/or administered Exchange Servers.

The steps in detail are as follows:

1.	The backup agent establishes a connection to and communication with the Microsoft Exchange Information Store service (MSExchangeIS).
2.	The checkpoint is frozen. New data continues to be written to the databases, but the Checkpoint pointer is not updated until the backup ends.
3.	The first log file that must be copied to the backup media is recorded in the database header in the Current Full Backup section.
4.	The copying of the databases begins. Changes during the backup that cannot be reconstructed fully from the log files are not flushed to disk. Instead, an extra file page is created and attached to the end of the .edb file as a mini header containing information relating to the transaction log files required to recover the database. This information can be seen in the Patch Current Full Backup section in a restored but not yet rolled forward database using ESEutil /MH from a command line. The information stored there overrides the Log Required field in the Database header.
5.	Because log files cannot be backed up while open, the current temp log file is closed and sequenced regardless of its size immediately after copying the database files.
6.	The transaction log files required to reliably restore the databases are now copied to the media. This includes all transaction log files such as those flagged in the checkpoint file and including the file just closed.
7.	Based on the Last Backup Date/Time information stored in the database headers together with the logs that were required and the information in the Checkpoint file, those transaction logs not required for a successful restore of the database are deleted from disk.
8.	The Previous Full Backup section of the database header is updated with the Date/Time and Log Range of the backup just completed.

Copy Backup

A copy backup is similar to the Normal backup with the exceptions that it does not delete the old log files and does not update the database headers to indicate that the backup has occurred. This can be useful for creating a rollback copy of the Exchange data prior to a restore or repair of the databases in the event that the restore or repair was unsuccessful.

Daily Backup

In Exchange Server 2003 a daily backup is the same as a copy backup.

Incremental Backup

Unlike normal and copy backups where the current transaction log file is closed and sequenced after the databases have been copied, with incremental backups, it is rolled over at the beginning. All the sequentially numbered log files are then copied to the media, and those not required for successful database recovery are deleted for disk. The Current Incremental Backup section of the database header is then updated with the Date/Time and log range of the backup just completed.

For an incremental backup to be useful in a disaster recovery you must have a normal or copy backup containing the ESE databases and an unbroken sequence of transaction log files.

Differential Backup

A differential backup is similar to an incremental backup with the exception that the old transaction log files are not deleted, and the Current Incremental Backup section of the database header is not updated.

Offline Backup

Offline backups are file-level backups made while the Exchange 2003 services are shut down or the data stores dismounted.

It is highly recommended that you do not use offline backups except in special cases immediately prior to recovering databases. Their one advantage is that because they do not check data integrity during the backup process they can be performed in cases where an online backup may fail due to data corruption.

To ensure the integrity of the data and transactions, including those in memory and as yet uncommitted to disk, you should use online backups.

Security Permissions

The user account that you are logged in to must have the requisite permissions or rights assigned when trying to back up or restore Microsoft Exchange data. Only those accounts with Domain Level Backup Operator rights can back up Exchange 2003 databases. To restore Exchange 2003 backups, the account must have full Exchange administrator rights for the domain.

Table 13.2 lists the minimum account levels needed for backup and restore.

Table 13.2. Minimum Account Levels for Backup and Restore

Operation	Minimum Account Level
Exchange backups	Domain backup operator
Exchange restore operations	Full Exchange administrator
Windows backups	Local backup operator
Windows restore operations	Local administrator rights

Note that you can assign users Domain Backup Operator permission without granting them full administrator rights. You can also use Run As to perform operations such as scheduled jobs in a security context other than that of the logged-on user.

Table 13.3 lists the group memberships and backup and restore privileges assigned to various security groups.

Table 13.3. Group Memberships and Backup and Restore Privileges

Group Membership	Backup Privileges
Local Administrators group	Members can back up most files and folders on the computer where the account is a member of the Local Administrators group. If the computer is an Exchange 2003 member server, you cannot back up Exchange database files unless you are also a member of the Backup Operator or Domain Administrator group.
Domain Administrators group	Members can back up all files and folders on all computers in the domain.
Local Backup Operators group	Members can back up all files and folders on the computer where the account is a member of the Local Backup Operators group.
Domain Backup Operators group	Members can back up all files and folders on all computers in the domain.
Any other domain or local group	Members can back up all files and folders that the account owns. Members can back up files or folders for which the account has Read, Read and Execute, Modify, or Full Control permissions.

Backing Up Exchange Using the SBS Backup Wizard

By default, the SBS Backup Wizard automatically configures and schedules Microsoft NTBackup to perform a full, online, Exchange-aware backup of the server, which includes the Exchange stores and logs together with Windows System State and user data.

There is no need (and in fact no option) to select the Exchange stores and logs when configuring the Backup Wizard. If you open the wizard-generated backup file in NTBackup, as shown in Figure 13.9, you will see that it included the Exchange Information Store First Storage Group. This is saved as C:\Program Files\Microsoft Small Business Server\Backup as Small Business Server Backup Script.bks in a default SBS install.

Figure 13.9. SBS wizard-generated backup.

By default SBS 2003 hides the Exchange Installable File System (IFS) drive, previously seen as the M virtual drive. If this is visible for any reason, you should add the entire drive to the list of file exclusions in the Backup Wizard.

At the Storage Allocation for Deleted Files and Email screen, shown in Figure 13.10, you can configure the period of time Exchange retains deleted emails and mailboxes before flushing them from the system. Users and administrators can then recover these items during the retention period directly from disk without recourse to the backup media.

Figure 13.10. Storage allocation for deleted files and email.

Before setting the retention period, consider the following issues:

• The volume of email

• The size of the email

• The size of the database and logs on disk

• The effect this will have on performance

• The effect on the size and duration of the backup

• The effect on the free space available on disk for a recovery or repair

• The effect that this will have on the duration of a restore

One thing to be aware of is that although the wizard-created backup uses the Volume Shadow Copy service in part, the Microsoft Exchange Server 2003 Writer for the Volume Shadow Copy service can conflict with NTBackup. If the Exchange Writer is enabled, it precludes the capability to simultaneously back up Exchange stores and System State. Microsoft has therefore disabled the Exchange Writer as part of the SBS setup.

Using NTBackup

The SBS Backup Wizard configures a restorable backup set that includes System State as well as your Microsoft Exchange data stores and is the easiest and recommended way to back up SBS. The wizard is a wrapper around NTBackup and a number of other services, but you can configure these manually if you want.

You can configure and run NTBackup independently depending on your plan and needs using either the GUI in the wizard or manual mode, or scripted via the command line.

Windows Server 2003 Backup uses Volume Shadow Copy services (VSS) to back up System State, but backs up the Exchange Server stores directly. The System State Backup Shadow Copy Provider changes the state to Backup in Progress, and other processes cannot then access the Information Store.

Exchange 2003 SP1 supports Volume Shadow Copy when used with backup tools that include an Exchange Server 2003 aware Volume Shadow Copy Service Requestor. Due to the lack of such a requestor in NTBackup you need to configure the backup independently of a System State backup. Some third-party software does include the requestor, however, and can back up System State and Exchange stores simultaneously using the Volume Shadow Copy service.

Not all Exchange data that may be required for a successful restore is backed up with the System State. Exchange relies on IIS for its SMTP component and for Web Services for Outlook Web Access (OWA). Exchange stores the configuration data in Metabase Stores in IIS. By default, the data is stored in two files in the C:\WINDOWS\system32\inetsrv\MetaBack folder.

The .MDn file, where n is a version number, holds the metadata, whereas the .SCn file holds the schema. You can create a one-time backup of these files using the GUI from the IIS Manager by selecting computer name, Action, All Tasks, Backup/Restore Configuration (as shown in Figure 13.11); then select Create Backup. You can then include these files in your Exchange backup, as shown in Figure 13.12.

Figure 13.11. Back up IIS metadata.

Figure 13.12. IIS metadata backup files.

As stated previously, you should not back up the Exchange Installable File System (IFS, M) drive if it is visible. Neither should you back up the database or log files unless the databases are dismounted and offline. Although it is strongly recommended that you use only online Exchange-aware backups, you may have reason to do an offline backup. In that event, dismount both the Private and Public Folder Stores before backing them up. You should also exclude the directories containing the databases and the log files (by default, C:\Program Files\exchsvr\mdbdata\) from selection.

It will be necessary to configure properties in the public and private folders that the SBS Backup Wizard would normally configure automatically. These include

• Circular logging

• Deleted item retention

• Deleted mailbox retention

• Do not permanently delete items and mailboxes until the store has been backed up

Using ExMerge

The Exchange Server Mailbox Merge Wizard (ExMerge) is a powerful tool used to extract and import information from and to Exchange private mailboxes using Outlook .pst files.

Since its creation in 1997 as a tool to remove Melissa virus-infected messages, it has grown into a sophisticated multithreaded application with considerable search and filtering capabilities on single or multiple mailboxes in a store.

ExMerge can extract and copy, move, or delete messages by selecting or excluding folders by specific subject, by attachment name, or by date/time range.

It can be used in a one- or two-step process from a GUI interface or command line and supports logging and scripted calls to a configuration (.ini) file.

Best Practice: ExMerge

Download the latest version of ExMerge appropriate to the version of Exchange and install it to the %Program Files%\Exchsrvr\bin directory. Add the %Program Files%\Exchsrvr\bin directory to the Systems Path variable so that ExMerge can locate required Exchange DLLs, and you can easily execute both it and other utilities such as ESEutil and ISinteg from the command line. Prepare yourself by reading the comprehensive manual. Add ExMerge to your toolkit and practice using it.

The latest version of ExMerge supports Outlook Calendar, Contacts, Journal, Notes, Tasks, Views, and Folder rules.

Although ExMerge works with both the First Storage Group and the Recovery Storage Group, it cannot extract data from Public Folders nor can it handle all data and metadata. Its search and filtering cannot find a string in a substring or an attachment to a message nested within another message.

ExMerge requires Receive As and Send As permission for the mailboxes to be able to import and export information. The logged-on user Account must have Service Account Administrator permissions at the organization, site, and configuration levels of the Administrator program and have both permissions on the mailboxes.

ExMerge is ideally suited to creating brick level (individual mailbox) backups and archives. Although it cannot write data directly to backup tape, it can to other media such as internal or external HDD or NAS that can be programmatically moved to tape. It can also create incremental backups by either of the following:

• MergeCopy only new messages and folders skipping all messages and folders previously copied.

• ReplaceReplace data only if the copy in the source store is more recent.

ExMerge uses a sophisticated process that helps recover all uncorrupted data even if individual mailbox folders contain corrupted messages. All messages in a mailbox folder are extracted collectively to minimize remote procedure call (RPC) traffic and time. On encountering an error, ExMerge automatically skips the message and then begins copying the messages that remain in the folder individually. After reaching the end of the folder, the tool then resumes copying messages collectively again until another error is encountered. This makes it an ideal recovery tool.

Another feature of ExMerge is that it is highly scriptable. You can create sophisticated .ini files (either manually or by saving your setting to files as seen in Figure 13.13). You can then script calls to the appropriate file for a variety of backup and disaster recovery operations.

Figure 13.13. ExMerge Change Settings Filenames window.

Using Third-Party Solutions

Although not specifically necessary, it's possible and common to use third-party backup solutions with SBS and Microsoft Exchange. These vary in price, feature set, and usability from those that are simple wrappers around the NTBackup software to those that fully support the backup APIs and VSS.

Third-party backup solutions can provide support for features that the SBS NTBackup does not, such as disk imaging, optical media, brick level backup and restore, and Microsoft Exchange backup via Volume Shadow Copy Services.

Additionally, there are vendor and service provider solutions that include offsite backup to remote storage backup via broadband and snapshot or streaming backup to NAS or hot spare servers.

In evaluating backup software for SBS and Microsoft Exchange 2003, consider the following:

• Price (cost/value benefit).

• Use of and support for MS Backup APIs (Exchange aware).

• Use of and support for Volume Shadow Copy services (VSS aware).

• Proprietaries and support for the backup hardware/medium.

• Whether it allows for online or offline only backup and restore.

• The duration of the backup and restore scenarios and methods.

• Reliability and appropriateness of the solution.

• The degree of difficulty.

• Documentation.

• Vendor support and longevity in the marketarchived backups may be many years old.

If you are going to use VSS aware backup software to back up your SBS Exchange databases, you need to turn on the Exchange Writer. You can do this by editing the Registry value located in HKLM\SYSTEM\CurrentControlSet\Services\MSExchangeIS\ParametersSystem and setting the Disable Exchange Writer value to a Value Data of 0. Either toggle the value to 1 or delete the Registry entry when no longer required.

Note

Be aware that enabling the Exchange Writer inappropriately can cause system errors and data loss. Make sure that the backup solution you are using supports it.

Caution

The usual warning about editing the Registry applies. Great care needs to be taken. You should have a known good backup of the Registry and know how to restore it in the event of disaster or lockout.