7.7 Managing storage groups

---

The Store process controls all of the storage groups that are active on a server. Exchange 5.5 uses a single client instance to the Store, used by the Store process to access the Private and Public Stores. Exchange 2000/2003 uses the client access mechanism to access the Store and Site Replication Services (SRS). However, these applications run in the context of separate processes, such as SRSMAIN.EXE, so they do not affect the Store. For Exchange, each storage group is a client instance running inside the Store process. As a separate instance, each storage group has its own set of transaction logs and checkpoint files.

The basic architectural concepts set down for the Store, including the atomic transaction model, logging, pointers, and use counts, are still valid for multiple storage groups. Operating multiple databases brings its own set of challenges, and we have to expand the established concepts to incorporate how storage groups work.

Figure 7.17 illustrates how storage groups help to make the current version of Exchange more resilient than earlier versions. Two storage groups are active on our sample server. If a problem affects the disk holding the EDB4 database, then you must take Storage Group 2 offline to fix the problem. Storage Group 1 remains active, and users whose mailboxes are located in any of the databases in Storage Group 1 will not be aware that a problem has occurred. You must take a complete storage group offline to fix a problem, even if a hardware problem only affects a single database. This is because the Store captures data for all of the transactions for the storage group into a single set of log files.

Figure 7.17: Storage groups.

Splitting the Store into multiple databases makes Exchange more tolerant to database failure at system startup too. The Exchange 5.5 Information Store service will refuse to start if it detects any problem with either the Private or the Public Store. The problem could be minor, and it is frustrating to find that an important service will not start up until everything is perfect. Once Exchange introduced storage groups, additional granularity was required inside the Store. Beginning with Exchange 2000, if Exchange detects a problem with a database, the Store marks the database as "offline" and continues to load the next database. The problem only affects the mailboxes or public folders held in the offline database. When the problem is fixed, you can bring the repaired database back online to restore full service.

7.7.1 Store status

You can put an Exchange database into two different states:

• Mounted: The database is online and available for use. This is the state during normal operation. The Store will not attempt to manage a database unless it is mounted.

• Dismounted: The database is offline and unavailable to clients. This is the state when you need to perform certain maintenance operations, such as a database rebuild or hard recovery, on databases. Exchange is able to start the Information Store service without mounting any databases, which might be the case during a restore operation.

You mount and dismount databases using ESM, as shown in Figure 7.18. You can also use the Computer Management console (Manage Applications and Services) to manage databases, and you can set a property of a database so that Exchange will not attempt to mount it as the Information Store service starts up (Figure 7.19).

Figure 7.18: The option to dismount a Store.

Figure 7.19: Do not mount a Store at startup.

Exchange maintains a set of parameters to control the functioning of the Store. You can find the global parameters in the system registry at HKLM\ Software\Microsoft\ESE98\Global\System Parameter Overrides. However, be very careful about making any changes to these parameters unless you are sure of what you are doing. Exchange holds the parameters for a storage group as properties of the storage group. Global parameters, applied to all storage groups, include the size of the cache allocated to the buffer manager and the version Store. Storage group properties include the locations for databases, transaction logs, and temporary databases.

7.7.2 Planning storage groups

It is unwise to rush to create additional storage groups and databases unless you have good reason to extend the Store. For example, you might want to isolate important users into a separate database where they have increased mailbox quotas. Alternatively, you might decide to split a Mailbox Store to reduce backup time after it reaches a specific size, such as 50 MB. Of course, if database size prompts you to create a new Mailbox Store, you need to move some mailboxes to the new Store to prevent the old Store from growing further, and you may want to run the ESEUTIL utility to rebuild the old Store and return space to the file system.

Exchange servers that host multiple companies may find it convenient to create multiple storage groups. With Exchange 5.5, all the companies share a common Information Store, but it is much better to be able to provide each company with its own SG or database, allowing you to create a storage firewall between each company's information.

Even the largest server is unlikely to use more than 20 databases in the near future, so it seems reasonable to come up with some rules of thumb that you can factor into planning exercises for Exchange deployments. For example, it is reasonable to pick a storage limit as the point at which you will consider partitioning the Store. In my experience, the average size of databases in production today on medium to large servers (500 mailboxes and upward) is still under 20 GB, so perhaps it is reasonable to say that 25 GB is a good starting point at which to consider creating a new Mailbox Store. The new Mailbox Store will still be in the default or first storage group. You could then decide that you will create a new storage group when an existing group contains three databases.

Alternatively, if you want to distribute I/O load and create additional resilience from disk failure, you might decide that you will create a new storage group after you have two fully populated 25-GB Mailbox Stores. Each storage group increases the administrative complexity of the system, especially in terms of backup and restore, so only consider moving to multiple storage groups if you have more than 50 GB to manage. As you add storage groups and databases to a server, be sure to use a reasonable naming convention to identify the storage entities and their directories. Figure 7.20 shows the convention used at HP for storage groups and databases. Note the way that the databases are associated with the storage groups through naming and the way that the database type is indicated in the name ("Mbx" = mailbox). To make it easier to identify important files using Windows Explorer, you can carry this naming convention forward into the physical file and folder names used for storage groups. Table 7.6 outlines a suitable naming convention. We use drive L: for transaction logs and S: for the Store databases to create a logical association between the drives and their purpose. Again, it is best to use the same drive letters for the same files on all servers.

Figure 7.20: Storage naming convention.

Table 7.6: Naming Convention for Storage Groups

Storage Group 1	File and Folder Names
Transaction Logs	L:\Exchsrvr\SG1_TransactionLogs\
Database Folder	S:\Exchsrvr\SG1_Mdbdata\
Mailbox Store 1	S:\Exchsrvr\SG1_Mdbdata\SG1MBX1.EDB
Streaming Store 1	S:\Exchsrvr\SG1_Mdbdata\SG1STM1.STM
Mailbox Store 2	S:\Exchsrvr\SG1_Mdbdata\SG1MBX2.EDB
Streaming Store 2	S:\Exchsrvr\SG1_Mdbdata\SG1STM2.STM

Service-level agreements are another point to consider, since an SLA may call for a recovery time for database outages that can only be satisfied by distributing mailboxes across multiple databases and perhaps multiple storage groups.

Every storage group and database that you add to a server makes demands on system resources. Databases take up disk space, and every storage group maintains its own set of transaction logs. Windows uses approximately 10 MB of physical memory to load the internal structures (tables and indexes) for each database into memory. Apart from the physical memory, Windows needs contiguous chunks of virtual memory to map the Store's internal structures. Large servers are normally equipped with plenty of memory, but it is still important to realize that loading up multiple databases will occupy resources from an unexpected quarter. For example, a server that splits processing across ten databases requires 100 MB of physical RAM just to start up. Windows and the other Exchange components also require memory and might consume 500 MB just to get a server up and running.

For maximum resilience and I/O performance, you should distribute each set of transaction logs to a separate set of disk spindles. As with Exchange 5.5, it is essential to separate the transaction logs from their associated databases to ensure that a hardware failure on a disk will not affect logs and databases at the same time. Protect each of the spindles used for transaction logs with RAID 1. There is no need to use RAID 5 to protect transaction logs, since this will just slow down I/O performance.

You affect Exchange's single-instance storage model when you deploy multiple databases. If you send a message to three recipients, and each of the recipients' mailboxes is located in a different Store, then the Store must create a separate copy of the message for each recipient. Obviously, the potential to create multiple (somewhat redundant) copies of messages is one reason why you want to restrain the urge to create multiple databases unless you have good reason to do so.

Exchange 2000 supports two-node (on Windows 2000 Advanced Server) or four-node active/active clustering (on Windows 2000 Datacenter Server). Exchange 2003 supports up to eight-node clustering. Clusters treat storage groups as resources, so they can move or fail over storage groups from a failed node to run on the remaining active nodes in the cluster. Clusters will run multiple storage groups, but you have to take care to ensure that the individual nodes in the cluster are able to support the load of all the storage groups in the case of failure. For example, in a two-node cluster you should not stress each server past 50 percent of its rated performance, because it will have a double workload if the other server fails. In the same way, use up to 66 percent of the rated performance of a server in a three- node cluster or 75 percent of rated performance in a four-node cluster. The last thing you want is for a server to collapse under the load when everything transitions to it after its cluster partners fail.

7.7.3 Creating new storage groups

Creating a new storage group is very straightforward. Indeed, Microsoft has made the operation so simple that the danger exists that an unwary system administrator will create new storage groups or databases without going through the necessary planning process. You can do all of the work to create a new Store or storage group through ESM, as shown in Figure 7.21.

Figure 7.21: Creating a new storage group from ESM.

When you create a new storage group, you provide a name and a location for its transaction logs (Figure 7.22). Place the logs on physical disk volumes separate from their databases. Servers that support several thousand mailboxes are likely to be equipped with multiple controllers and many physical disks, and you should take every step to place files in such a manner that a failure on any individual drive will not affect multiple databases or sets of transaction logs.

Figure 7.22: Specifying details of a new storage group.

Always disable circular logging for storage groups that support mailboxes. However, it is possible to have circumstances where you would want to have circular logging turned on. Let us assume that you have a server that accepts an NNTP newsfeed from many different Internet newsgroups. The traffic to newsgroups can be very heavy, which results in a large and rapid buildup of transaction logs as the Store replicates newsfeed content to public folder replicas on other servers. Newsgroups typically store transient information that is removed after a short interval, typically 7 to 14 days, so the need to recover transactions if a database is restored is clearly not as high as for mailboxes or regular public folders. In this situation, it is quite acceptable to create a new storage group specifically to host newsgroups and enable circular logging.

Note the checkbox to control "zero out deleted database pages" in Figure 7.22. Some companies wish to ensure that there is no possibility to recover message content after users delete items and empty their deleted items folder. If this checkbox is set, Exchange will overwrite the pages that contain deleted messages with a bit pattern that effectively zeros out deleted pages to remove any chance that anyone could ever look at the data again. Zeroing out pages places a little extra overhead on the system, so it is best to leave the checkbox alone unless you really need to use this feature. It is best practice to perform a full backup after enabling this feature, since the biggest impact of page zeroing occurs during backup time when ESE actually writes the pattern to zero database pages.

After the Store creates the new storage group, you can proceed to create the Stores that the storage group will manage. Figure 7.23 shows the prop- erties for a new Mailbox Store before creation. Note that each database has a separate maintenance schedule for activities such as background defragmentation. This is to ensure that background maintenance does not swamp a server by running maintenance tasks for multiple databases concurrently. Background maintenance tasks are less likely to affect multi-CPU servers, but it is still something to keep an eye on. Figure 7.23 clearly shows the locations of the two files that make up the new Mailbox Store. The first is the familiar EDB database that Exchange has used since 4.0. The second is the name and location of the streaming file that holds Internet content. Remember, the Store creates a streaming file for every EDB database.

Figure 7.23: Viewing database locations for a new Store.

Often, administrators find that they need to relocate store databases to a new volume. Perhaps you have just commissioned a SAN and want to move databases over to use the new storage, or maybe you have just increased capacity on one of the existing volumes. In whatever circumstances, ESM has to dismount Mailbox and Public Stores before it can move them. You can also dismount the stores manually by selecting the Store through ESM and then selecting the "dismount" option, but it is easier to let ESM do all the work.

To move a Store, select it through ESM and access its properties. Go to the database property page and use the browse button to select a new location for the EDB and streaming file files, and then click "OK" or "Apply" to make the move. ESM then warns you that the move affects existing backups, because ESE can no longer apply data in the transaction logs from the backups to the database after the move, if you need to recover transactions after a hard restore (Figure 7.24). For this reason, you should always take a full backup immediately after you move a database.

Figure 7.24: Warning before moving a database.

---