The Architecture of Message Queuing 3.0

I l @ ve RuBoard

If you're familiar with MSMQ running under Windows NT and Windows 2000, you should be aware that the architecture of Message Queuing 3.0 under Windows XP and Windows .NET Server includes some subtle changes.

Queues, Servers, and Active Directory

The Message Queuing 3.0 architecture is based on queues, Message Queuing 3.0 Servers, and Message Queuing 3.0 Clients. Message Queuing 3.0 Servers and Clients (capitalized) are services that run on Windows XP and Windows .NET Server computers; they are distinct from the client and server (lowercase) applications that are built using them.

Applications developed using Message Queuing 3.0 submit messages to queues or receive messages from queues. A queue can be public or private. A public queue is created and managed by a single computer, and the details of the queue (but not its messages) are published in the Active Directory database. Active Directory is managed by the domain controller, which is usually also a Message Queuing 3.0 Server. Other computers in the same domain can use the Message Queuing 3.0 APIs to query Active Directory and obtain information about the public queues in that domain. Applications can submit messages to public queues, subject to any security that is enforced. Depending on where the queues are located, messages might be routed through a Message Queuing 3.0 Server.

Messages sent by a Message Queuing 3.0 Client are temporarily cached in local outgoing queues until they have been transmitted. Outgoing queues are created automatically as needed, and they disappear when they're no longer required ”you cannot create or delete them yourself. If a Message Queuing 3.0 Client computer becomes detached from the network, it can still be used to send messages, but the messages will be retained in an outgoing queue and transmitted only after a connection to the domain is reestablished. The Message Queuing 3.0 Server can also provide routing services to queues in other domains and remote sites, although it's better if routing servers are not installed on primary domain controllers. (Use a secondary domain controller instead.)

Note

Message Queuing 3.0 Client computers in the same domain can have direct connectivity with each other (if they use the same network protocol) and can bypass the Message Queuing 3.0 Server when they send messages to each other, but ideally you should configure Message Queuing 3.0 on these computers to send messages through the Message Queuing 3.0 Server for routing. This can help to control the flow of messages around the network and reduce the amount of bandwidth needed, by concentrating the message traffic.

Of course, the use of public queues depends on all participating computers being members of a domain. A computer that is not part of a domain cannot create or access a public queue and must use private queues instead. (This restriction also applies when you use a workgroup.) Like a public queue, a private queue is created and managed by a single computer, but its details are not published in Active Directory. A computer cannot access a private queue on a second computer unless the first computer knows the name of the private queue and has been granted appropriate access rights by the second computer. Computers that are not part of a domain must have a direct connection to one another because routing facilities will not be available either.

You can use private queues in a domain environment if you need to keep queue information secret.

Transactional Message Queues

When a message is posted to a queue, you can never be quite sure when it will be received. In some applications, you might want to post a message, or a series of related messages, to a queue as part of a single greater atomic operation. You'll also want to ensure that if something goes wrong partway through the operation, any messages already posted can be "unsent" ”that is, removed from the queue before being received. Achieving this manually can be a taxing process that involves locking and synchronizing access to the queue along with any other resources involved in the operation. Fortunately, this requirement occurs so often that Microsoft has provided a built-in a mechanism to support it: transactions.

A message queue can be marked as transactional. You probably associate transactions with updates to databases, but the same principle can be applied to any operation that reads and writes data. In the case of a message queue, an application can start a transaction that reads the message from the queue and processes it, and then the application can commit the transaction. In the event of a failure, the transaction will be rolled back and the message will be returned to the head of the queue, ready to be read again.

Transactions can be complex, involving the sending and receiving of multiple messages in conjunction with modifications to other resources such as a database. In the event of a rollback, every send and receive operation performed and every modification made during the transaction must be undone. The Message Queuing 3.0 services work in conjunction with the Microsoft Distributed Transaction Coordinator (MS DTC) to achieve this.

Transactions consume resources, so you should mark a queue as transactional only if it is absolutely necessary.

Managing Queues

You can create and manage queues using code (as described later in this chapter) or manually using the Computer Management snap-in, which is available under Administrative Tools in Control Panel. The Message Queuing service is listed in the Services And Applications folder. You can use the Action menu commands to create, view, and delete messages and message queues when the appropriate folder or queue is selected.

Figure 12-1 shows the Message Queuing configuration on one computer. There's a single public queue called CakeQueue (what else?), as well as a pair of private queues called msmqtriggernotifications (which was created automatically and handles message queuing triggers) and MyQueue (which I created manually).

Figure 12-1. The computer management console showing the Message Queuing configuration

If you have the Professional or Enterprise edition of Visual Studio .NET, you can also use Server Explorer to create, examine, and modify message queues, as shown in Figure 12-2.

Figure 12-2. Server Explorer in Visual Studio .NET displaying message queues

System Queues

Besides public and private queues, the Message Queuing 3.0 architecture also uses a number of system queues.

The system queues you're most likely to notice are the journal queues and the dead-letter queues. Each computer that's configured to support Message Queuing 3.0 will contain a single outgoing journal queue, which is sometimes referred to as the computer journal queue . (It's in the System Queues folder, under Message Queuing in the Computer Management console). When a message is sent from the computer, a copy can be retained in this queue, specified on a message-by-message basis. Each message queue, private or public, also has an associated journal queue that records copies of messages received on that queue. This feature is disabled by default but can be enabled on a queue-by-queue basis. Each journal is configured with a finite amount of storage. Once this storage is exceeded, new messages will no longer be journaled. (No errors will occur, however.) It's up to the administrator to purge the journal queues periodically.

The dead-letter queues hold messages that failed to be delivered because the destination queue did not exist or the message expired , because it was not delivered within a reasonable time, or because of some other unforeseen error that rendered the message undeliverable. There are two dead-letter queues in the System Queues folder, which are often referred to as the computer dead-letter queues . The Dead-letter messages queue holds failed nontransactional messages, and the Transactional dead-letter messages queue holds failed transactional messages. Again, an administrator should monitor and purge these queues periodically.

Message Delivery

Messages posted to a queue can be assigned a priority that determines how quickly they'll proceed to the head of the queue to be read by a receiving application. The priority can be set to a value between 0 and 7. The value 0 indicates the lowest priority, and 7 is the highest priority (the default is 3). You can specify the priority when you submit a message. Transactional messages always have the same priority (0) because they must be received in the same order in which they were sent, and any priority specified by the developer will be ignored.

Queues can also be assigned a priority between “32768 and 32767. (The default is 0.) Queues with a higher priority will be serviced by the message queuing software in the operating system to deliver messages to applications before those of a lower priority. You should be careful when you modify queue priorities because a low-priority queue can easily be starved by a group of higher-priority ones.

Messages themselves can be delivered to a message queue using one of two modes: express or recoverable. An express message is stored in memory during routing and delivery and consequently can be processed very quickly. The drawback is that if a system failure occurs that causes a machine to shut down, any express messages in the memory of that computer will be lost. Express messages are resistant to network failure, however ”they'll simply sit in the memory of the routing server that's attempting to deliver them to the destination until the network connection is restored.

Recoverable messages are written to temporary files on disk as they're transmitted and are thus tolerant to many forms of machine failure and premature shutdown. When the computer restarts, the message queuing service can resume processing recoverable messages. Recoverable messages are also resistant to network failure.

You specify the delivery mechanism as part of the message. (The default is to use recoverable delivery.) Only nontransactional messages can be marked for express delivery; transactional messages must be recoverable.

Message Queuing Triggers

Message queues can be monitored using message queue triggers. A message queue trigger allows you to specify automatic processing when messages are submitted to a queue. A trigger analyzes messages that appear on the queue by applying one or more rules. A rule can specify a match based on criteria such as the message priority, the message contents, or the source of the message. If a rule matches the message, an action indicated by the trigger will be executed. An action can invoke a COM component or run a standalone executable. The trigger itself can leave the message on the queue for a receiving application or remove it, acting as a filter.

You can apply message queue triggers to system queues such as the computer journal queue, as well as to ordinary public and private queues.

I l @ ve RuBoard