9.3 The transport core

---

The transport core is the heart of Exchange message routing. It is broken down into four major components:

• Advanced Queuing:   This component manages messages as they enter and exit the different queues within the transport core. The queuing engine supports domain-level queues, where messages are grouped according to their final destination (such as any message sent to recipients at hp.com), as well as link queues, where messages are grouped according to the next hop that they will be routed across to get to their final destination. For example, messages sent to hp.com might have to travel across a link to an ISP. As explained later in this chapter, Exchange maintains destination queues for messages sent to domains such as hp.com, as well as link queues for messages waiting to travel on the next link to their final destination. For example, if you send messages to hp.com via a connection to an ISP, Exchange maintains a link queue for the ISP and places any messages going to hp.com there along with messages for any other address that the ISP handles. A message can pass from server to server and move from link queue to link queue, but it will stay on the same domain-level queue. The queuing engine is supported by the QueueAdmin API, which is used by the Exchange System Manager snap-in to view queue status. You can use this API to stop and start queues as well as to enumerate queue contents. The Advanced Queuing component is also responsible for generating delivery service notifications.

• Categorizer:   The categorizer analyzes message headers and properties to decide how to best process messages. Senders and recipients are resolved against the AD to ensure that Exchange respects any administrator-imposed restrictions and that it can route messages to addresses in the headers. A limited version of the categorizer is included in the basic Windows SMTP server to enable it to expand mail-enabled groups. The version provided with Exchange 2003 adds support for features such as checking for mailbox quotas, whether a mailbox can receive mail from a specific sender, which Mailbox Store a mailbox is located in, whether a user can send messages across a specific connector, and so on.

• Exchange Store Driver:   This is the interface between the Advanced Queuing component and the NTFS file system. Windows ships with a basic Store driver, which allows it to save incoming SMTP messages as formatted files; these are later processed to accomplish tasks such as AD replication. The Exchange Store driver is an upgraded version, which includes ExIFS support; this enables Advanced Queuing also to have direct access to messages in the Store.

• Routing Engine:   This component replaces the RID master and GWART used to determine routing paths within an Exchange 5.5 organization. It is a much more sophisticated routing mechanism and is capable of assessing incoming updates about the state of the network and adjusting the path messages will take. Exchange can route messages according to their size, the sender of a message, and their priority. This information is taken into account, together with the cost associated with the various links defined between servers in the organization and the current state of the links, to decide how a message is eventually routed.

Exchange uses a modified version of Dijkstra's algorithm to determine routing decisions. The method used to route messages is explained in section 9.7. Figure 9.6 illustrates how the different parts of the transport core work together. While the diagram looks complicated at first appearance, the detailed workings of the transport core present a far more complicated picture than that depicted here. However, it is sufficient to reveal the essential workings of Exchange's transport core. To explore how the parts of the Routing Engine work together, we can follow the path of a message as the Routing Engine processes it within the transport core:

Figure 9.6: The Exchange transport core.

• Messages can enter the transport core from the SMTP service, the MTA, or the Store. Messages coming from the SMTP service originate from other Exchange servers or external systems and arrive as NTFS files, which are then processed via ExIFS. MAPI mailboxes submit messages through the Store, and messages arriving from Exchange 5.5 servers come in through the MTA. IMAP and POP3 clients submit messages through the SMTP protocol stack in IIS, which, in turn, uses ExIFS to create the message bodies and insert them into the streaming file. HTTP-DAV clients can access ExIFS directly without going through the SMTP protocol stack. Message properties are promoted into the appropriate Mailbox Store. In all cases, the messages are placed into the Inbound Queue.

• The queuing engine manages processing of the inbound messages and passes them to the Message Categorization code. Message Categorization performs checks such as mailbox quota limits for recipients and verifies that the message can be routed onward. The individual addresses in distribution groups are expanded at this time.

• The message goes on to the Categorized Message Queue. The transport core now checks to see whether any routing event should be triggered for the message. For example, if an event implements a check to ensure that outgoing messages should not contain profanities, it would be possible to scan the text of the message and its attachment. The message could then be routed onward or returned to the originator if some problem words were found. Routing events are designed to facilitate the fastest possible delivery of messages to the final destination, so checking for profanities is not a particularly good example of how a routing event is used. Nevertheless, it does demonstrate that events open up all sorts of new possibilities to developers.

• A lookup is performed against the Domain Mapping and Configuration Table. This table tells the transport core to route the message for local delivery or to one of the SMTP virtual servers that are active on the system.

• Messages intended for local recipients are placed on a local delivery queue. "Local" means anything that is not another SMTP server (including remote Exchange servers). Remember that the MTA is still tightly integrated with the Store, so any message that must be dealt with by the MTA has to be routed through the Store. At this point, another event could be triggered before the message is actually delivered to a mailbox. For example, a copy of the message could be sent to a special recipient if it came from a user in a particular department. The message is then passed to the Exchange Store driver, which either delivers the message to a local mailbox or passes it to the MTA for onward transmission to an Exchange 5.5 site or legacy connector.

• Messages for other SMTP servers are allocated to the destination queues for the SMTP virtual servers allocated to handle messages for the different domains. The Connection Manager monitors the links to other SMTP servers for the destination message queues and adjusts routing as required. For instance, if the link state table were updated to reflect the fact that a hub server was currently not available, the Connection Manager might reroute messages that pass through the affected server via another hub. To improve message throughput, Exchange 2003 performs DNS resolution for SMTP addresses synchronously instead of the asynchronous processing done by Exchange 2000.

• All outgoing messages are dispatched to their final destination using the standard Windows SMTP service.

The introduction of query-based distribution groups can pose a huge processing challenge for the Routing Engine, which must resolve the LDAP query to build the complete recipient set represented by a query-based group before it can decide how to route a message. Consider a query that generates 400,000 recipients (a real-life example). The two biggest problems are the time taken to generate the list (between six to eight hours on a fast server) and the fact that Exchange does not checkpoint the recipient set, so the Routing Engine has to start over if any problem prevents it from generating the complete list. The combination of Exchange 2003 (which fetches 1,500 recipients at a time in LDAP queries instead of 1,000 for Exchange 2000) and Windows 2003 provides the fastest performance for query-based distribution groups, but even so, you should arrange to submit messages addressed to groups that expand into very large collections at a time of low user demand. One further issue is that there is no way for an administrator to track the progress of list expansion, so once a message is submitted, it goes into a black hole until it comes out again. Message tracking can help you follow the course of a message through the transport core. Exchange 2003 upgrades message tracking to capture even more information than before.

9.3.1 Domain and link queues

Exchange uses two different queues in the final stages of routing SMTP messages. The Routing Engine maintains queues on a per-domain basis. In other words, all the messages for a specific SMTP domain (such as *.com) are held together on one queue. The Routing Engine takes messages off domain queues and places them on link queues, which Exchange maintains on a per-hop basis. This means that the Connection Manager moves messages to the most suitable link queue for final routing. You can equate queues to links in some respects, but it is not 100 percent accurate to think that both are the same. A link is an object that is associated with multiple queues, each of which can provide messages that will use the link. Another way of thinking of this is to define a link as an outbound connection to a single host that can be used by multiple queues.

Per-destination queues allow Exchange to handle hold-ups in message delivery to specific destinations. The Exchange 5.5 IMS implements a different solution by holding all messages in a single queue until they are sent. If problems are encountered transmitting messages to a domain, then all messages are blocked until the problem is resolved. For example, let's assume that the IMS is used to connect Exchange 5.5 sites in Dublin, London, Dallas, and New York. Messages normally go from Dublin to Dallas via New York. If the network link goes down between Dublin and New York, messages are rerouted if another route exists. If not, they remain on the queue until the link is fixed. This underscores the importance of factoring in multiple available routes between routing groups, if possible, when you consider the design of an Exchange organization. Note too that only the messages on a single link are ever blocked by a failure that closes off communication to a specific destination. Messages going to all other destinations continue to flow unless they have to pass along the link affected by the failure.

The Routing Engine will know that the link between Dublin and New York is down and will attempt to discover alternate routes. If links are available between Dublin to London, and London to Dallas, then messages can be moved to the per-link queue for London and be routed via that location. Rerouting happens automatically and does not have to be manually reconfigured when problems occur.

---