Message Queuing Basics

Database products such as Microsoft SQL Server divide information into multiple databases that contain multiple tables. Message queues separate information into distinct queues that are usually associated with different operations or tasks. For example, an e-commerce program might read order information from a ProductOrders queue, and a data-mining component might read from a RequestedTasks queue. By default, all queues are stored in the directory C:\[Windows Dir]\System32\Storage of the host computer. Outgoing messages that haven't yet been sent might be stored in memory or they might also be found in the Storage directory on the sending computer, depending on the requested transmission method. Figure 8-1 shows a simplified diagram of Message Queuing.

Figure 8-1. Message transmission with Message Queuing

Types of Message Queues

To perform its work, Message Queuing requires several types of message queues. The following sections introduce each one briefly.

Public Queues

Public queues are published in an Active Directory listing and are replicated across a Windows 2000 domain. You can browse or search for a public queue without knowing the name of the computer that hosts it (and therefore you can also move a queue from one computer to another without affecting the client).

Private Queues

Private queues can be used in a workgroup environment without a dedicated Message Queuing server, they don't support authentication, and they can't be located without the computer name. These queues are used most often for development testing before a full deployment or for small-scale use.

Outgoing Queues

Outgoing queues are local internal queues that are used to store messages before they are sent to the destination (remote) queue. Messages can be stored for long periods of time in an outgoing queue if the computer is off line, and they will be automatically sent when connectivity is reestablished. Outgoing queues are generated automatically and can't be created or deleted manually.

Connector Queues

Connector queues reside on Message Queuing servers. They act as a proxy for queues provided by other computers and managed by other messaging systems. For example, you can use Microsoft's MSMQ-MQSeries Bridge (included with the Host Integration Server 2000 product), which allows your .NET applications to interact with IBM MQSeries queues. Similarly, you can use the Microsoft Message Queuing Connector (MQC), which allows connectivity with computers running Message Queuing on non-Windows-based platforms such as UNIX or AS/400. Connector queues handle message transition seamlessly and allow an application to use a foreign queue in the same way it uses an ordinary Message Queuing queue. Without connector queues, there is no interoperability between different Message Queuing products.

Journal Queues

Journal queues contain copies of messages that have been sent or processed, for future reference. There are two types of journaling: source and target.

When sending a message, the sender can elect to use journaling to store a copy of the message on the current computer. Source journaling is therefore controlled on a per-message basis. Target journaling is set on a per-queue basis. If a destination queue has journaling enabled, it stores a copy of every message it receives in a separate journal queue.

Dead-Letter Queues

Dead-letter queues contain messages that expired without being successfully delivered. In some cases, the message might have expired after being forwarded to another computer (or even arriving at the target), in which case it is placed in the dead-letter queue of the computer where it was when it expired. Messages for transactional queues (discussed later in this chapter) are stored in a separate transactional dead-letter queue.

What makes a message undeliverable? A number of factors can intervene, including the following:

• The queue is full, cannot be found, or is not known to the sender.

• A nontransactional message was sent to a transactional queue, or vice-versa.

• The message exceeds its maximum number of allowed hops or its maximum allowed lifetime. For example, it might have exceeded the time-to-reach-queue (TTRQ) limit (which is set by the sender) or its time-to-be-received (TTBR) limit (which the recipient sets).

Administration Queues

Administration queues just store acknowledgement messages (for example, messages that confirm that another message was received). They shouldn't be confused with internal private queues, which Message Queuing uses behind the scenes to manage other queues and record diagnostic information. This chapter does not discuss internal queues.

Report Queues

The report queue is used to track the route a sent message takes if message route tracking is enabled. There can only be one report queue per computer.

The Message Queue Service

If you're new to Message Queuing, you're probably wondering about the underlying technology that drives it. Basically, Message Queuing uses a Windows service to dispatch and receive messages. You can find the service listed, like all others, in the Computer Management utility (as shown in Figure 8-2). This service actually has a fair bit of responsibility, including reading from queues, writing to queues, and working across the network with other Message Queuing services to route messages to their destination.

Figure 8-2. The Message Queuing service

Configuring Message Queues

You can perform every Message Queuing task, including creating and removing queues, from a .NET application. In the majority of cases, however, you will create your queues manually with the Computer Management tool and then use .NET code just for sending messages. This is much the same way you would use a server-side database.

You can configure queues by expanding the Message Queuing node under the Services And Application node. You can right-click on a queue category folder to create a new queue for your application. You'll be prompted to enter a queue name and indicate whether the queue is transactional. Transactional queues ensure that messages arrive in the same order they were sent (and hence the message priority is ignored). Transactional queues also guarantee that messages arrive only once. Transactional message queues do not enlist the message sending and message receiving into a single atomic operation, as you might assume. Even with transactional queues, Message Queuing uses a disconnected model in which send and receive operations are decoupled from one another.

After creating a queue, you can right-click and modify its properties (as shown in Figure 8-3).

Figure 8-3. Message queue properties

The options you can configure include the following:

• Label

  A string that can identify the queue to allow easy searching.

• Type ID

  Enables you to group multiple queues into a category with a shared GUID. You can generate a GUID using a tool such as guidgen.exe or uuidgen.exe; these are included with the .NET Framework.

• Maximum Size

  Ensures that you won't fill a disk with messages. Otherwise, Windows XP allows unlimited message storing, and Windows 2000 allows up to 2 GB of messages.

• Privacy Level

  Determines whether the message will be encrypted. Optional indicates that the sender can choose, whereas Body indicates that unencrypted messages will be rejected. Note that message queue encryption requires Active Directory.

• Journal

  If this option is enabled, a journal queue is created for this queue with copies of all received messages. This is primarily useful for debugging purposes. You can also limit the size of the journal queue.

Finally, it's worth noting that you can perform all the same tasks for creating and managing queues using the Server Explorer window in Visual Studio .NET (as shown in Figure 8-4).

Figure 8-4. Configuring queues in Visual Studio .NET

The Anatomy of a Message

So far we've defined message queues as message lists, but we haven't really considered what a message contains. Every Message Queuing message consists of a few basic properties, such as a label (the title of the message), a priority (which determines the order in which the receiving application reads the messages, assuming they are nontransactional), and the body (which contains the actual message content). The message body is not limited to textual data. Instead, it can include any type of data formatted using binary or XML encoding. However, the recipient application assumes the responsibility of recognizing and reconstructing the appropriate object based on the message data. .NET can't enforce any type of compile-time type safety, which means that invalid messages will lead to runtime errors, which should be caught with exception handlers.

In addition, the message sender sets additional properties that configure the timeout and identify who sent the message. The message sender can also choose the delivery mode. With Express mode, messages are transmitted very quickly and serialized to memory until they can be sent to their network destination. With Recoverable mode, messages are stored in a file first, ensuring that even if the computer crashes no information is lost. When creating a client that might be disconnected from the network, you should use Recoverable mode. Otherwise, unsent messages are lost as soon as the operating system shuts down.

Are Message Queues Suitable for Cross-Platform Programming?

Message Queuing is a Windows operating system service. Non-.NET clients won't be able to make use of it directly. However, that doesn't mean that Message Queuing can't play a large part in a cross-platform application. Remember, part of .NET's strategy is to provide clients on other platforms with a facade through XML Web services (for application programmers) or ASP.NET Web pages (for application users). In a typical cross-platform system, you might use an XML Web service or Web page to receive an order and then send a message through a message queue, which will be read and processed by another .NET component. The message queue, like the database, plays an internal or "back-end" role and is never directly accessed by the client.