7.1 Structure of the Store

---

Figure 7.1 illustrates the internal structure of the Exchange Store, which is unchanged in Exchange 2003. The Store Kernel is the component that builds features unique to Exchange on top of the generalized Jet Blue engine. These features include single-instance storage, views, and automatic indexing of items as clients add items to the Store. The Jet Blue layer builds on top of the generalized ESE database engine. This layer takes responsibility for managing low-level internal structures such as the B-trees and class tables. ESE has no knowledge of high-level structures such as the schema used in any particular database. Instead, ESE operates at the page level to manipulate database contents.

Figure 7.1: The structure of the Store.

We have referred to Jet, which stands for Joint Engine Technology, a generalized Microsoft database engine that Microsoft builds off of to form variants that drive products as diverse as the Store and Access. ESE is a much-enhanced development of the basic Jet engine now used by the AD and the DS. Apart from Exchange and Access, there are many other examples of Jet in use within Microsoft products. For example, Windows use Jet databases to hold WINS and DHCP data, and these databases share many of the characteristics you see in Exchange, including transaction logs and reserved logs. The original plan was for Jet to evolve to become a common technology platform for Microsoft databases, but this never materialized. In the future, Microsoft wants to achieve the vision of a unified database platform through the Yukon engine. Future versions of Exchange will use a variant of Yukon, much as the current version of Exchange uses a variant of Jet.

By definition, ESE is a multiuser ISAM database with full Data Manipulation Language (DML) and Data Definition Language (DDL) capability. Applications such as Exchange use ESE to store and index records in an efficient manner and in different ways. Microsoft has optimized ESE for highly scalable performance through fast retrieval of data, with a priority given to accessing data held in memory caches instead of going to disk whenever possible. The massive numbers of simulated MAPI users supported on Exchange servers demonstrate just how scalable ESE is. Even if you would not want to connect 30,000 users to a single eight-way server, the fact that the database is able to cope with the load generated by so many simulated users is an impressive testament to its inherent scalability.

Microsoft refers to the version of ESE implemented in Exchange 2000 and 2003 as ESE98 to differentiate it from previous implementations, such as that used in Exchange 5.5, which share the same basic engine (ESE97) as the AD. However, there are some subtle differences between ESE97 and ESENT, the variant used by the AD, the most obvious of which are the different sizes of pages and transaction logs.

Because of the common link to Jet, it is easy to make the erroneous conclusion that Exchange and Access share the same database engine. This story has become one of the street fables around Exchange and surfaces every so often in Internet newsgroups. Microsoft designed the variant used by the Store to deal with the hierarchical nature of the mailbox/folder/item model found in messaging and document management systems. A hierarchical model is much more appropriate than a relational model for clients such as Outlook, which navigate to data using a tree structure similar to a file system. Internally, there are certainly many relationships maintained within the Store. Folders have a relationship to mailboxes, and subfolders have a relationship to their parent folders. Folders hold items, and users own mailboxes. The need to deal with loosely formatted semistructured data is another major difference between the classic database engines and ESE. One message transaction is a simple email sent to three users, but a user might send the next transaction to a massive distribution list, and the message could contain four attachments, each of which is larger than 5 MB. A database designed to handle structured transactions, such as those typical of financial applications, is not necessarily going to be able to handle the type of transactions seen in a messaging system.

---