MSMQ

[Previous] [Next]

MSMQ 2 is built into Windows 2000; it doesn't run on earlier versions of Windows. MSMQ 1 runs on Windows NT, Windows 95, and Windows 98. It's not difficult to achieve interoperability between the two versions of MSMQ if you already have Windows NT and Windows 98 computers running in a Windows 2000 domain.

MSMQ 2 relies on Windows 2000 Active Directory to maintain a global catalog of configuration data. The MSMQ catalog spans a forest of Windows 2000 domains and tracks profiles of MSMQ-related entities such as computers, public queues, and security-related certificates. You can say that each MSMQ catalog defines an MSMQ enterprise.

The MSMQ Enterprise

To create an enterprise of MSMQ computers, you must first install MSMQ on a Windows 2000 computer that's been configured as a domain controller with Active Directory installed. A Windows 2000 domain controller is often called an Active Directory server. This first installation of MSMQ on an Active Directory server triggers the creation of MSMQ's enterprise catalog in Active Directory. A Windows 2000 forest can have multiple Active Directory servers with MSMQ installed. Each Active Directory server has a copy of the MSMQ catalog.

After you install MSMQ on an Active Directory server, it's pretty easy to install MSMQ on other computers in order to add them to the same MSMQ enterprise. You simply have to make sure that the other computers have been added to a domain in the same forest as the first Active Directory server before you install MSMQ.

The type of deployment I've just described is known as MSMQ 2 enterprise mode. You can also set up a group of computers in MSMQ 2 workgroup mode, which allows you to use MSMQ without needing a Windows 2000 domain controller or Active Directory. However, MSMQ 2 enterprise mode makes the MSMQ network far more capable. For this reason, this chapter will cover MSMQ only as it applies to computers that use Active Directory in a Windows 2000 domain.

You can configure MSMQ in a few different ways when you install it on a computer that's not an Active Directory server. You can set up an MSMQ computer as a routing server, an independent client, or a dependent client. A routing server can have its own queues. A routing server, as its name implies, can also act as a store-and-forward agent for messages targeted at a queue on another MSMQ computer. This is helpful when client applications can't send their messages directly to the target server. The routing server acts as an intermediate hop between the computer sending a message and the target computer, which holds the destination queue. Note that an Active Directory server with MSMQ installed performs all the same tasks as a routing server. It simply holds a copy of the MSMQ catalog in addition to these other responsibilities.

You can also configure an MSMQ computer as an independent client, which doesn't perform store-and-forward duties but can create and manage its own local queues. This is a good installation choice for a remote computer such as a laptop because a client application can send messages to a network queue even when the laptop is disconnected. In such a situation, MSMQ creates a temporary local queue behind the scenes and caches messages as they're sent. Later, when the independent client computer reestablishes a connection to the network, the MSMQ infrastructure automatically forwards the messages to the destination queue.

A dependent client can't maintain queues locally. It can't function if it's disconnected from the network. It must be able to establish a connection to an Active Directory server or a routing server in order to send and receive messages. A desktop computer that's always connected to the LAN is typically the only type of machine that you'd run as a dependent client.

MSMQ embraces the concept of a Windows 2000 site, which is a set of computers on one or more subnets that all have high-speed access to one another. A site is usually a set of computers running on the same LAN. Each Windows 2000 site that contains MSMQ computers should have one or more Active Directory servers so that you can run queries against the MSMQ catalog without having to call across sites. Active Directory automatically maintains a copy of the MSMQ catalog on each domain controller and keeps this cached data up-to-date through replication.

You should understand that sending messages across sites is slower and less reliable than sending messages between two computers in the same site. MSMQ allows administrators to assign costs to each intersite link. MSMQ can then run an algorithm to determine the optimal routing for sending messages from one site to another. This administrative detail is an important aspect of tuning an MSMQ enterprise when computers are sending messages across sites that span large geographical distances.

The queue manager

The communication infrastructure of MSMQ relies heavily on a component known as the queue manager. The queue manager runs as a Windows service and is launched from MQSVC.EXE. MSMQ installs a queue manager on every computer that has store-and-forward responsibilities or that has its own local queues. The only type of MSMQ installation that doesn't get its own queue manager is a dependent client.

Each queue manager is assigned a unique GUID at installation time. MSMQ uses this GUID to track queues that are created and owned by a specific queue manager. It also uses this GUID to track messages that originate from the queue manager. It's important to realize that a computer never has more than one queue manager. This means that a queue manager's GUID also identifies the computer on which it's installed. MSMQ tracks this GUID in Active Directory along with the information required to resolve the computer address when necessary.

In short, queue managers make possible all communication in MSMQ. A sender application must transmit a message to a queue manager. If the queue manager is the owner of the destination queue, it stores the message in the queue. If it doesn't own the destination queue, it forwards the message to another queue manager. If the computer with the destination queue can't be reached, the queue manager temporarily stores the message and forwards it later when communication can be established. Finally, receiver and reader applications communicate with the queue manager on the computer that holds the queue in order to receive or peek at messages.

I've already mentioned that MSMQ maintains a global catalog in Active Directory. Each queue manager also maintains additional data for local queues in a place called the local queue store (LQS). The LQS consists of a file for each local queue and configuration data that the queue manager uses to manage its local queues. Note that the LQS often duplicates information that's in Active Directory. The point of caching duplicate information is that the queue manager can use the LQS to eliminate network round trips and to handle messages at times when it can't connect to an Active Directory server.

MSMQ Programming

You can program with MSMQ in two ways. MSMQ exposes a C-level API as well as a COM-based component library. As a Visual Basic programmer, you're much better off using the component library when you program with MSMQ. Note that some MSMQ functionality is accessible only through the C-level API. Fortunately, you can meet most of the common requirements for message passing in a distributed application by using the component library. In the rare circumstance in which you need to use the C-level API, you can use Visual Basic Declare statements and call into the API directly. This chapter will examine the use of MSMQ's component library only. If you include a reference to MSMQ's type library (the Microsoft Message Queue 2.0 Object Library) in your Visual Basic project, you can use the MSMQ components.

MSMQ provides several components, each of which has quite a few properties and methods. I can't cover every aspect of programming with MSMQ's component library. I'll simply offer some programming examples that demonstrate the most common types of code required in a distributed application.

I'll assume that you have MSMQ installed on a Windows 2000 computer that's configured as an Active Directory server, a routing server, or an independent client. In addition to allowing you to write and test MSMQ programs, this type of MSMQ installation allows you to use the Computer Management administrative tool to carry out some basic MSMQ administrative tasks. This tool (shown in Figure 10-4) lets you manage and examine queues and messages. It's accessible via the Administrative Tools group, which is in the Programs group on the Windows Start menu.

Figure 10-4 The Windows 2000 Computer Management administrative tool lets you create and manage local queues.

Creating a public queue

Let's begin our tour of MSMQ by creating a queue. You can create public queues and private queues. You can create a public queue in one of two ways—manually, using the Computer Management administrative tool, or programmatically.

Let's create a public queue manually. After you launch the Computer Management administrative tool, you simply right-click the Public Queues folder and choose Public Queue from the New menu. You must give the queue a name and specify whether you want the queue to be transactional. I'll describe transactional queues later in this chapter—for now, simply create a queue by giving it a name and leave the Transactional check box cleared. Click the OK button to create the queue.

After you create the queue, you can examine its attributes by right-clicking it and choosing Properties. You'll see a tabbed dialog box in which you can modify various properties of the queue. You'll notice that MSMQ automatically generates a GUID to serve as the ID for the queue. The GUID associated with a public queue is known as the instance ID.

To create a queue programmatically, you can use an MSMQQueueInfo object. First, you create the object and assign it a valid PathName, which should include the name of the computer and the name of the queue. For example, the following code uses an MSMQQueueInfo object to create a new queue:

 Dim qi As MSMQQueueInfo Set qi = New MSMQQueueInfo qi.PathName = "MyComputer\MyQueue" qi.Label = "My Queue" qi.Create 

Once you set the PathName property, you can create a queue by invoking the Create method. This example also sets the Label property of the new queue. A label is optional, but it can be helpful when you need to locate the queue later on.

The Create method has two optional parameters. The first parameter indicates whether the new queue is transactional. The second parameter, IsWorldReadable, indicates whether the queue will be readable to users other than the queue's owner. The default value for this parameter is False, which means that only the queue's owner can receive messages from the queue. If you pass True instead, the queue can be read by all users. Whatever value you pass, all users can send messages to the queue. You can also set queue security permissions by modifying the discretionary access control list (DACL) for the queue. You do this by opening the queue's Properties dialog box and navigating to the Security tab in the Computer Management administrative tool.

Note that you can abbreviate the PathName for a local queue so that you don't have to hardcode the name of the computer. You must do this if you want to write generic code that will run on many different computers. A single dot (as in .\MyQueue) signifies that the queue path is defined on the local computer. You can use this abbreviated form when you create and open a local queue. For example, you can rewrite the previous code as follows:

 Dim qi As MSMQQueueInfo Set qi = New MSMQQueueInfo qi.PathName = ".\MyQueue" qi.Label = "Market Order Submission Queue" qi.Create 

You can also use an MSMQQueueInfo object to search for or open an existing queue. You run a query against Active Directory with an MSMQQuery object to determine whether a queue with a certain label exists. You run the query by invoking the LookupQueue method, which returns an MSMQQueueInfos object. The MSMQQueueInfos object is a collection of MSMQQueueInfo objects that match your lookup criteria. Here's an example of conducting a lookup by a queue's label:

 Dim qry As MSMQQuery Set qry = New MSMQQuery Dim qis As MSMQQueueInfos Set qis = qry.LookupQueue(Label:="Market Order Submission Queue") Dim qi As MSMQQueueInfo Set qi = qis.Next If Not (qi Is Nothing) Then     ' Open queue and send order message. Else     ' Degrade gracefully if queue cannot be located. End If 

In this example, a client runs a query against the MSMQ catalog in Active Directory to locate a queue with a well-known label. This dynamic lookup approach can provide flexibility because a client application doesn't have to know the name of the computer on which the queue resides. You can thus move queues around the network without having to rewrite applications.

Now let's open a queue and send a message. The next object you need to understand is an MSMQQueue object. The relationship between MSMQQueueInfo objects and MSMQQueue objects can be a little confusing. It's reasonable to conclude that an MSMQQueue object represents a physical queue because of its name. However, you're better off thinking of it as a queue handle. For example, you can open three different MSMQQueue objects on the same physical queue:

 Dim qi As MSMQQueueInfo Set qi = new MSMQQueueInfo qi.PathName = ".\MyQueue" Dim qSend As MSMQQueue Set qSend = qi.Open(MQ_SEND_ACCESS, MQ_DENY_NONE) Dim qPeek As MSMQQueue Set qPeek = qi.Open(MQ_PEEK_ACCESS, MQ_DENY_NONE) Dim qReceive As MSMQQueue Set qReceive = qi.Open(MQ_RECEIVE_ACCESS, MQ_DENY_NONE) 

You can see that an MSMQQueueInfo object represents a physical queue and that an MSMQQueue object actually represents an open handle to the queue. When you call Open, you must specify the type of access you want in the first parameter. When you open a queue with MQ_RECEIVE_ACCESS, you can peek at messages in addition to receiving them. However, if you want to send messages while also peeking at or receiving from the queue, you must open two MSMQQueue objects. Remember to invoke the Close method on an MSMQQueue object after you finish using it.

You can use the second parameter of Open to specify the share mode for the queue. The first possible setting for this parameter is MQ_DENY_NONE, which means that the queue can be opened by more than one application for receive access at the same time. You must use this setting when you open a queue using MQ_PEEK_ACCESS or MQ_SEND_ACCESS. However, when you open a queue with receive access, you can set the share mode to MQ_DENY_RECEIVE_SHARE to prevent other applications from receiving messages at the same time. When one application opens a queue with both MQ_RECEIVE_ACCESS and MQ_DENY_RECEIVE_SHARE, no other application can open the queue in receive mode. An application using this mode is the only one that can remove messages from the queue.

Creating a private queue

When you create a public queue, MSMQ assigns it an instance ID (a GUID) and publishes it in Active Directory. This allows other applications to open the queue by assigning the computer name and queue name to the PathName property. This also allows an application to locate the queue from across the network by running a query with an MSMQQuery object. However, the process of publishing a public queue takes up time and disk space and is sometimes unnecessary.

Imagine an application that consists of hundreds or thousands of independent clients that all require local response queues. In this situation, it makes sense to use private queues. Private queues must be created locally, and they're not published in Active Directory. They're published only in the LQS of the computer on which they reside.

As you'll see later in this chapter, you can send the return address for a private response queue in the header of a request message so that the receiver application can return a response message. You can thus establish bidirectional communication between a client application and the server. More important, using private queues means that you don't have to publish all those response queues in Active Directory, which saves both time and disk space.

You can create a private queue by adding Private$ to the queue's PathName, as shown here:

 Dim qResponseInfo As MSMQQueueInfo Set qResponseInfo = New MSMQQueueInfo qResponseInfo.PathName = ".\Private$\MyResponseQueue" qResponseInfo.Create 

MSMQ applications can send messages to private queues on other machines as long as they can find the queues. This isn't as easy as locating public queues because you can't open a private queue using a PathName—it isn't published in Active Directory. Later in this chapter, I'll show you a technique for passing the response queue's information to another application in a request message.

Locating a queue

Both public queues and private queues are assigned instance IDs by MSMQ at creation time. Each queue carries the same instance ID for its entire lifetime. When you open a queue using PathName or some other technique, a queue manager must ultimately resolve the instance ID before you can access the queue.

MSMQ generates an instance ID for a private queue using a different format from an instance ID for a public queue. As you know, MSMQ generates a unique GUID to serve as an instance ID for a public queue. A private queue's instance ID, on the other hand, is made up of the unique GUID of the local queue manager and a computer-specific queue ID. Here's an example of an instance ID for a private queue.

 f38f2a17-218e-11d2-b555-c48e04000000\00000022 

If you know the instance ID of a queue, you can open the queue using the FormatName property instead of the PathName property. Here's an example of two different FormatName properties, one for a public queue and one for a private queue:

 PUBLIC=067ce2cb-26fc-11d2-b56b-f4552d000000 PRIVATE=f38f2a17-218e-11d2-b555-c48e04000000\00000022 

Once you know how to generate the FormatName property, you can open a private queue on another computer using the following code:

 Dim MyFormatName As String MyFormatName = "PRIVATE=f38f2a17-218e-11d2-b555-c48e04000000\00000022" Dim qi As MSMQQueueInfo Set qi = new MSMQueueInfo qi.FormatName = MyFormatName Dim qSend As MSMQQueue Set qSend = qi.Open(MQ_SEND_ACCESS, MQ_DENY_NONE) 

In some cases, you can open a queue using either FormatName or PathName. But when you open a private queue, you can't use PathName. A private queue doesn't have an instance ID that's published in Active Directory. As you saw in the previous example, however, you can send a message to a private queue across the network by assigning the proper FormatName before invoking the Open method. Using FormatName rather than PathName is preferable when the target queue's information isn't published in Active Directory or when an Active Directory server isn't accessible.

Your final option for accessing a queue is using a direct format name, which specifies a protocol, a machine address, and a destination queue name. When you use this technique, you rely on the destination computer's LQS for queue name resolution. Here are a few examples of a direct format name:

 DIRECT=TCP:209.27.34.23\MyQueue DIRECT=TCP:209.27.34.23\private$\MyResponseQueue DIRECT=OS:MyComputer\MyQueue 

You typically use direct format names only as a last resort, when you can't use other techniques. For example, you must use direct format names to enable communication between two MSMQ computers that have been set up in workgroup mode. You must also use direct format names to enable communication across MSMQ enterprises. In other words, you must use direct format names to send messages between two computers that don't share the same Active Directory catalog.

The use of direct format names poses a major limitation. MSMQ isn't capable of temporarily storing a message on a routing server as the message passes from the sender to the recipient. The messages must be sent directly between the sender computer and the computer with the target queue. This means that the two machines must be able to establish a direct connection and that they can't benefit from the distributed store-and-forward mechanism that's built into the infrastructure of MSMQ.

Sending a message

Let's send our first message. MSMQ makes this task remarkably easy. You create a new MSMQMessage object and prepare it by setting a few properties. You then invoke the MSMQMessage object's Send method, and MSMQ routes your message to its destination queue. Here's a simple example of sending a message to a queue across the network:

 Dim qi As MSMQQueueInfo Set qi = New MSMQQueueInfo qi.PathName = "MyServerComputer\MyQueue" Dim q As MSMQQueue Set q = qi.Open(MQ_SEND_ACCESS, MQ_DENY_NONE) ' Create a new message. Dim msg As MSMQMessage Set msg = New MSMQMessage ' Prepare the message. msg.Label = "My superficial label" msg.Body = "My parameterized request information" msg.Priority = MQ_MAX_PRIORITY ' Send message to open queue. msg.Send q q.Close 

As you can see, MSMQ's component library makes it easy to open a queue and send a message. The message in the example above has three properties set. Label is a string property of the message header that distinguishes or identifies a particular message. The two other message properties are the Body and Priority.

In MSMQ, a message body is stored as an array of bytes. The body is typically used to transmit parameterized data between the sender and the receiver. Our example shows that you can simply assign a Visual Basic string to a message body. The receiver can read this string from the message body just as easily. However, in many cases you'll use a message body that is more complex. For example, you might need to pass multiple parameters from the sender to the receiver. I'll revisit this topic in Chapter 12 in the section on data passing and describe your options for packing parameterized information into the body of an MSMQ message.

The last property used in the example specifies the message priority. A message has a priority value between 0 and 7; the higher the value, the higher the priority. MSMQ stores messages with higher priority levels at the head of the queue. For example, a message with a priority level of 6 is placed in the queue behind all messages of priority 7 and behind messages of priority 6 that have already been written to the queue. The new message is placed ahead of any messages of priority 5 or lower. The MSMQ type library contains the constants MQ_MAX_PRIORITY (7) and MQ_MIN_PRIORITY (0). The default priority for a new message is MQ_DEFAULT_PRIORITY (3).

Before I move on, I want to introduce a few other important message properties. When a message is sent to a queue, MSMQ assigns it an ID property. The ID property is a 20-byte array that uniquely identifies the message. MSMQ generates the ID by using two different values. The first 16 bytes of the ID are the GUID of the local queue manager. This means that the first part of the message ID is the same for all messages sent from the same computer. The last four bytes of the ID are a unique integer generated by the queue manager. In most cases, you don't need to worry about what's in the byte array. You should just assume that every ID is unique. However, if you need to compare two IDs to see whether they represent the same message, you can use VBA's StrComp function with the vbBinaryCompare flag.

Each message also has a CorrelationID property. Like the ID, this property is stored as a 20-byte array. Let's look at an example to see why this property is valuable. Let's say that a client application sends request messages to a server. The server processes the requests and sends a response message for each request. How does the client application know which request message is associated with which response message? The CorrelationID property takes care of this.

When the server processes a request, it can assign the ID of the incoming request message to the CorrelationID of the outgoing response message. When the client application receives a response message, it can compare the CorrelationID of the response message with the ID of each request message. This allows the sender to correlate messages. As you can see, the CorrelationID is useful when you create your own response messages. MSMQ also assigns the proper CorrelationID automatically when it prepares certain system-generated messages, such as acknowledgment messages.

Receiving and peeking at messages

To receive a message, you first open an MSMQQueue object with receive access and then invoke the Receive method to read and remove the first message in the queue:

 Dim qi As MSMQQueueInfo Set qi = New MSMQQueueInfo qi.PathName = ".\MyQueue" Dim q As MSMQQueue Set q = qi.Open(MQ_RECEIVE_ACCESS, MQ_DENY_NONE) Dim msg As MSMQMessage ' Attempt to receive first message in queue.  Set msg = q.Receive(ReceiveTimeout:=1000) If Not (msg Is Nothing) Then     ' You have removed the first message from the queue.     MsgBox msg.Body, vbInformation, msg.Label Else     ' You timed out waiting on an empty queue. End If q.close 

There's an interesting difference between sending and receiving a message with MSMQ. You invoke the Send method on an MSMQMessage object, but you invoke the Receive method on an MSMQQueue object. (This doesn't cause any problems; it's just a small idiosyncrasy of the MSMQ programming model.) If a message is in the queue, a call to Receive removes it and returns a newly created MSMQMessage object. If there's no message in the queue, a call to Receive behaves differently depending on how the timeout interval is set.

By default, a call to Receive has no timeout value and will be blocked indefinitely if no message is in the queue. If you don't want the thread that calls Receive to be blocked indefinitely, you can specify a timeout interval. You can use the ReceiveTimeout parameter to specify the number of milliseconds that you want to wait on an empty queue.

If you call Receive on an empty queue and the timeout interval expires before a message arrives, the call to Receive returns with a null reference instead of an MSMQMessage object. The code in the previous example sets a timeout value of 1000 milliseconds. It also determines whether a message arrived before the timeout expired. If you don't want to wait at all, you can set the ReceiveTimeout value to 0. A ReceiveTimeout value of -1 indicates that you want to wait indefinitely. (This is the default if you don't pass a timeout value.)

You can call Receive repeatedly inside a Do loop to synchronously remove every message from a queue. The following example receives all the messages from a queue and fills a list box with message labels:

 Dim qi As MSMQQueueInfo Set qi = New MSMQQueueInfo qi.PathName = ".\MyQueue" Dim q As MSMQQueue Set q = qi.Open(MQ_RECEIVE_ACCESS, MQ_DENY_RECEIVE_SHARE) Dim msg As MSMQMessage Set msg = q.Receive(ReceiveTimeout:=0) Do Until msg Is Nothing     lstReceive.AddItem msg.Label     Set msg = q.Receive(ReceiveTimeout:=0) Loop q.Close 

You can set the share mode to MQ_DENY_RECEIVE_SHARE so that your application won't have to contend with other applications while removing messages from the queue. Use a timeout value of 0 if you want to reach the end of the queue and move on to other business as soon as possible.

Sometimes you'll want to inspect the messages in a queue before removing them. You can use an MSMQQueue object's peek methods in conjunction with an implicit cursor to iterate through the messages in a queue. After opening a queue with receive access or peek access, you can call Peek, PeekCurrent, or PeekNext.

Peek is similar to Receive in that it reads the first message in the queue, but it doesn't remove the message. If you call Peek repeatedly, you keep getting the same message. Another problem with Peek is that it has no effect on the implicit cursor behind the MSMQQueue object. Therefore, you'll use PeekCurrent or PeekNext more often than Peek.

You can move the implicit cursor to the first message in a queue using a call to PeekCurrent. As with a call to Receive, you should use a timeout interval if you don't want to block on an empty queue. After an initial call to PeekCurrent, you can iterate through the rest of the messages in a queue by calling PeekNext:

 Dim qi As MSMQQueueInfo Set qi = New MSMQQueueInfo qi.PathName = ".\MyQueue" Dim q As MSMQQueue Set q = qi.Open(MQ_PEEK_ACCESS, MQ_DENY_NONE) Dim msg As MSMQMessage Set msg = q.PeekCurrent(ReceiveTimeout:=0) Do Until msg Is Nothing     ' Add message labels to a list box.     lstPeek.AddItem msg.Label     Set msg = q.PeekNext(ReceiveTimeout:=0) Loop q.Close 

The ReceiveCurrent method is often used in conjunction with PeekCurrent and PeekNext. For example, you can iterate through the messages in a queue by peeking at each one and comparing the properties of the current message against criteria of the messages you want to receive and process. For example, after calling PeekCurrent or PeekNext, you can compare the label of the current message with a specific label that you're looking for. If you come across a message with the label you're looking for, you can call ReceiveCurrent to remove it from the queue and process it.

Receiving messages using MSMQ events

The preceding example uses a technique for synchronously receiving messages, which gives you an easy way to read and remove all the messages that are currently in a queue. It also lets you process future messages as they arrive at the destination queue. However, this approach also holds the calling thread hostage. If you have a single-threaded application, the application can't do anything else.

You can use MSMQ events as an alternative to this synchronous style of message processing. MSMQ events let your application respond to asynchronous notifications that are raised by MSMQ as messages arrive at a queue. You can therefore respond to a new message without having to dedicate a thread to block on a call to Receive.

Let's look at how MSMQ events work. The MSMQ eventing mechanism is based the MSMQEvent component. To use events, you must first create an MSMQEvent object and set up an event sink. Next, you must associate the MSMQEvent object with an MSMQQueue object that's been opened for receive access. You create the association by invoking the EnableNotification method on the MSMQQueue object and passing a reference to the MSMQEvent object. After you call EnableNotification, MSMQ notifies your application when a message has arrived by raising an Arrived event.

To create an event sink, you must use the Visual Basic WithEvents keyword and declare the source object's reference variable in the declaration section of a form module or a class module. The following code sets up an event sink for a new MSMQEvent object in a form module of a standard EXE project:

 Private RequestQueue As MSMQQueue Private WithEvents RequestEvent As MSMQEvent Private Sub Form_Load()     Dim qi As MSMQQueueInfo     Set qi = New MSMQQueueInfo     qi.PathName = ".\MyQueue"     Set RequestQueue = qi.Open(MQ_RECEIVE_ACCESS, MQ_DENY_NONE)     Set RequestEvent = New MSMQEvent     RequestQueue.EnableNotification RequestEvent End Sub 

Once you set up the MSMQEvent object's event sink and call EnableNotification, you'll be notified with an Arrived event when MSMQ finds a message in the queue. Here's an implementation of the Arrived event handler, which processes messages as they arrive in the queue:

 Sub RequestEvent_Arrived(ByVal Queue As Object, ByVal Cursor As Long)     Dim RequestQueue As MSMQQueue     Set RequestQueue = Queue  ' Cast to MSMQQueue to avoid late binding.     Dim msg As MSMQMessage     Set msg = RequestQueue.Receive(ReceiveTimeOut:=0)     If Not (msg Is Nothing) Then         ' Process message when it arrives.     End If     RequestQueue.EnableNotification RequestEvent End Sub 

Note that this example calls EnableNotification every time an Arrived event is raised. This is required because a call to EnableNotification sets up a notification only for the next message. If you want to receive notifications in an ongoing fashion, you must keep calling EnableNotification in the Arrived event. Also note that the code in the example receives every message that was stored in the queue when the MSMQEvent object was set up. In other words, MSMQ raises events for existing messages as well as future messages.

Creating listener applications using Visual Basic

I'm very fond of Visual Basic, but I must point out two limitations of this development tool when you use it to create server-side listener applications. First, in a mission-critical system it's best to deploy the listener application as a Windows service. This results in higher levels of availability. Second, if your listener application has to handle a high volume of incoming request messages, the best way to increase system throughput is to increase the number of worker threads that are receiving messages from the queue and processing requests. Typically, monitoring a queue with a thread pool of 5 to 20 threads results in the highest levels of throughput.

Some hardcore programmers have found ways to turn a Visual Basic application into a Windows service and to spawn additional COM-aware threads, but these are off-road techniques. They're not supported by Visual Basic and therefore result in code that can be pretty difficult to maintain.

Many companies use alternative approaches such as using C++ to create a custom listener application that exhibits maximum availability and throughput. Another approach is to leverage the multithreaded listener service that's built into the architecture of Queued Components. If you leverage the Queued Components service, you can get the multithreaded behavior while maintaining all your application code in Visual Basic. I'll explain how to do this later in the chapter.

You should also realize that you can sometimes get by with a server-side listener application that's a standard EXE project running on a single thread. You might not need to resort to C++ development or rely on Queued Components. This might be the case if you need to create a listener application that never receives more than a few messages per minute or a batch-processing application that runs once a day, at midnight, to process the messages that have accumulated in a request queue during the day. In such cases, a simple Visual Basic application can provide a much cheaper approach to creating a custom middle-tier listener application. As you'll see a little later, working in terms of raw MSMQ programming can also provide more options and flexibility than Queued Components.

Working with a response queue

When a client application sends a message to a request queue, it's often desirable to send a response message from the server-side listener application back to the sender. For example, a response message can contain a notification to inform the sender that the transaction ran successfully or that the transaction failed. A response message can also return a requested set of data back to the sender.

When you prepare a message in the sender application, you can add information to the message header that tells the listener application how to locate a response queue on the sender's computer. Each MSMQMessage object has a ResponseQueueInfo property. You must set up this property using an MSMQQueueInfo object. The receiving application can use ResponseQueueInfo to open the sender's response queue and send a receipt message. The code in Listing 10-1 shows how the listener application can receive a request message, open the sender's response queue, and send a response message.

Listing 10-1

 Dim qi1 As MSMQQueueInfo, RequestQueue As MSMQQueue Set qi1 = New MSMQQueueInfo qi1.PathName = "MyComputer\MyQueue" Set RequestQueue = qi1.Open(MQ_RECEIVE_ACCESS, MQ_DENY_RECEIVE_SHARE) Dim msgRequest As MSMQMessage Set msgRequest = RequestQueue.Receive(ReceiveTimeout:=0) If Not (msgRequest Is Nothing) Then     ' (1) Process message (implementation omitted for clarity).     ' (2) Send notification back to client (implementation below).     Dim qi2 As MSMQQueueInfo, ResponseQueue As MSMQQueue     Set qi2 = msg.ResponseQueueInfo     Set ResponseQueue = qi2.Open(MQ_SEND_ACCESS, MQ_DENY_NONE)     Dim msgResponse As MSMQMessage     Set msgResponse = New MSMQMessage     msgResponse.Label = "Receipt for: " & msgRequest.Label     msgResponse.Body = "Your sales transaction has been completed"     msgResponse.CorrelationId = msgRequest.Id     msgResponse.Send ResponseQueue     ResponseQueue.Close End If RequestQueue.Close 

When you create a listener application that returns response messages to the client application, you should typically assign the ID of an incoming request message to the CorrelationId of the outgoing response message. This technique is shown in Listing 10-1. When the client application receives a response message, it can compare the CorrelationId of the response message with the ID from each request message. This allows the sender to correlate messages.

You should also consider using private queues instead of public queues when you set up your response queues, especially in a distributed application with hundreds or thousands of clients. Unlike public queues, private queues are not published in Active Directory. Using public queues unnecessarily wastes processing cycles, network bandwidth, disk space, and administrative overhead.

Transactional Messaging

During the design phase, you should decide how you want MSMQ's queue managers to move your messages across the wire. You can choose a technique that's fast and less reliable or a technique that's slower but provides more guarantees. Performance and reliability are always at odds.

Each MSMQ message has a Delivery property. This property has two possible settings. The default setting is MQMSG_DELIVERY_EXPRESS, which means that the message is sent in a fast but unreliable fashion. Express messages are retained in memory only as they're moved across various routing servers toward their destination queue. If a computer crashes while holding express messages, the messages might be lost.

To ensure that a message isn't lost while being routed to its destination queue, you can set the Delivery property to MQMSG_DELIVERY_RECOVERABLE. The message is flushed to disk as it's passed from one computer to another. The disk I/O required with recoverable messages results in significant performance degradation, but the message won't be lost in the case of a system failure. When you send nontransactional messages, you must explicitly set the Delivery property if you want recoverable delivery. When you send transactional messages, the Delivery property is automatically set to MQMSG_DELIVERY_RECOVERABLE.

Even when you set the Delivery property to recoverable, you can still experience problems. A single message might be delivered to the destination queue more than once. In such a case, the listener application will process the same request multiple times, which can result in undesirable behavior. You can send messages in a more reliable fashion by using transactional queues.

When you send messages in a transaction, you get a few guarantees that you don't get with nontransactional messages. First, queue managers take extra precautions to ensure that messages aren't lost or duplicated. They also make sure that messages in a transaction are delivered in the order in which they were sent. Moreover, the sender can always determine when a message has failed en route to a destination queue.

Let's look at an example of in-order delivery. Let's say that you send message A and then you send message B to the same queue. Message A arrives before message B as long as they are part of the same transaction. This means that message A is placed closer to the head of the queue. However, other messages from other transactions might be interleaved with yours. MSMQ guarantees the ordering of messages only within a single transaction. Also note that due to these in-order guarantees, transacted messages can't be assigned different priorities. In MSMQ, every transacted message has a Priority value of 0.

MSMQ allows transacted messages to be sent only to transactional queues. You can't change the transactional attribute after a queue has been created. It is therefore important to indicate whether you want a queue to be transactional when you create it. You can use the first parameter of the MSMQQueueInfo component's Create method to indicate whether you want a transactional queue.

You can send and receive transactional messages with MSMQ in two ways. You can use MSMQ's internal transactioning mechanism, or you can use external transactions that are coordinated by the DTC using the two-phase commit protocol. Each technique offers distinct advantages.

MSMQ provides its own internal transactioning mechanism, which provides the best performance for transactional messaging. However, when you use internal transactions, MSMQ can't coordinate the transaction with any other type of resource manager, such as a SQL Server database. Internal transactions don't use the DTC. Instead, MSMQ uses a more efficient protocol that's tuned for transactional messaging. Consequently, internal transactions are much faster than externally coordinated transactions.

MSMQ supports the OLE Transactions protocol, which means that it's a valid resource manager that can participate in DTC-based transactions. You can use it along with other resource managers when you run declarative COM+ transactions. Your connection to a transactional queue can be enlisted with the DTC just like a connection to a SQL Server database. External transactions are also known as coordinated transactions because they're managed by the DTC.

While coordinated transactions are slower than internal transactions, you can incorporate message passing along with operations to other types of resource managers. For example, you can write a transaction that receives a request message, modifies a SQL Server database, and sends a response message. Because the three operations are part of a single transaction, the DTC enforces the ACID rules across all this writing activity. (See Chapter 8 for details about the ACID rules.)

MSMQ internal transactions

MSMQ provides a shortcut for sending a single transacted message: You pass the constant MQ_SINGLE_MESSAGE when you call Send. Here's what a call to Send looks like:

 msg.Send q, MQ_SINGLE_MESSAGE 

You'll want to send single messages in a transaction when you want exactly-once delivery. If you don't send a message in a transaction, the message is more likely to be lost or duplicated.

To send multiple messages in an internal transaction, you must create a new MSMQTransactionDispenser object. You can then invoke its BeginTransaction method to start a new transaction. A call to BeginTransaction returns an MSMQTransaction object. You can pass a reference to this MSMQTransaction object when you call a message-writing operation such as Send, Receive, or ReceiveCurrent. Here's an example that performs a receive operation on a transactional request queue while sending two messages to a transactional response queue:

 Dim td As MSMQTransactionDispenser Set td = New MSMQTransactionDispenser Dim tx As MSMQTransaction Set tx = td.BeginTransaction() Set msg1 = RequestQueue.Receive(Transaction:=tx)  msg2.Send ResponseQueue, tx  msg3.Send ResponseQueue, tx  ' Commit the transaction. tx.Commit 

Remember that messages sent to the same queue inside the same transaction have in-order delivery guarantees. In this example, you can assume that msg2 will be placed in the queue ahead of msg3. However, there's no guarantee that messages from other transactions will not be interleaved between them.

The timing with which transacted messaged are added to and removed from transactional queues is tricky and it requires some attention on your part. Messages aren't sent to the response queue until the transaction has been committed. While the process isn't overly intuitive, it's important to see that a transacted message hasn't been delivered to the destination queue at the time the transaction commits. That's just the point at which the message begins its journey from the sender's computer.

When you receive a message in a transaction, it's removed from the queue immediately. Another application looking at this queue doesn't see the message even though the transaction hasn't been committed. However, if the transaction is aborted, the message is returned to the queue. This behavior is the same for transactional queues whether you're using internal transactions or external transactions.

You must follow a few rules when you program against transactional queues. If you violate one of these rules, MSMQ raises a runtime error.

• You can't send a message to a nontransactional queue from within a transaction.
• You can't send a message to a transactional queue if the message isn't part of a transaction.
• You can't receive and send messages to the same queue from within the same transaction.
• You can receive transactional messages only when the receiving application is running on the same computer as the transactional queue.

Another thing to keep in mind is that MSMQ transactions run at a lower isolation level than those in a DBMS such as SQL Server. As you'll recall from Chapter 8, SQL Server by default conducts the operations in a COM+ transaction with an isolation level of Serializable. MSMQ doesn't support the Serializable isolation level because of the way it manages locks during a transaction.

MSMQ provides an isolation level of Repeatable Read for a receiving application when other applications are sending messages to the same transactional queue. For example, the receiver might start a transaction and see the queue as empty. Later, within the same transaction, the receiver application might see phantom inserts made by other transactions. If MSMQ were to run transactions with an isolation level of Serializable, it would have to place an exclusive lock on the queue and block every send operation until the receiving transaction was complete. A locking scheme such as this would pose an unacceptable concurrency restraint in a queuing system.

When two transactions are run by two different receiving applications, however, the isolation level can be Read Uncommitted. For example, let's say that a single message is in a transactional queue. If transaction A receives the message from the queue, transaction B sees the queue as empty. If transaction A later aborts, the message is written back to the queue. The queue was really never empty, but transaction B saw it as empty because it read the queue in an uncommitted state.

MSMQ's inability to run serializable transactions isn't a flaw. Other messaging products, such as MQSeries, exhibit the same behavior. It's just an inherent problem with transactional queuing. Isolation is sacrificed to maintain higher levels of concurrency. As a result, you can't make as many assumptions as you can when you work with a DBMS that can run its operations with an isolation level of Serializable.

External transactions with MSMQ

The MSMQ type library contains a component for creating DTC-based transactions called the MSMQCoordinatedTransactionDispenser. This component's interface is identical to that of the MSMQTransactionDispenser component that you saw earlier in this chapter. Both components expose a single method, BeginTransaction, which returns an MSMQTransaction object. In fact, in most of your code you can use these two components interchangeably.

The difference between the two components is that the coordinated dispenser works with the DTC, while the internal dispenser doesn't. When you commit an external transaction, the DTC executes the two-phase protocol against MSMQ, which acts as a standard resource manager. The coordinated dispenser also lets you enlist a resource manager other than MSMQ into a transaction. However, Visual Basic offers no straightforward way to enlist another resource manager such as SQL Server using the MSMQTransaction object. You must explicitly enlist a database connection with a low-level call to ODBC or OLE-DB. You should probably avoid doing this in most production code written with Visual Basic. This means that most Visual Basic programmers should avoid using the coordinated dispenser directly.

Fortunately, you can get the same benefits of external transactions by sending and receiving messages from within a COM+ transaction. The COM+ runtime automatically enlists MSMQ as well as other resource managers such as SQL Server and Oracle into a declarative transaction. Here's an example that sends a message from a configured component that requires a transaction:

 ' Assume that this component requires a transaction. Dim oc As ObjectContext Set oc = GetObjectContext() ' (1) Write to a SQL Server database. ' (2) Send a message. Dim qi As MSMQQueueInfo Set qi = New MSMQQueueInfo qi.PathName = "MyComputer\MyQueue" Dim q As MSMQQueue Set q = qi.Open(MQ_SEND_ACCESS, MQ_DENY_NONE) Dim msg As MSMQMessage Set msg = New MSMQMessage msg.Body = "The database has been updated" msg.Send q, MQ_MTS_TRANSACTION q.Close ' (3) Commit the transaction. oc.SetComplete ' or call SetAbort. 

You can pass the constant MQ_MTS_TRANSACTION in a call to Send, Receive, or ReceiveCurrent to indicate that you want your writing operation to be part of the current COM+ transaction. You don't actually have to pass this parameter because this value is the default. Note that passing MQ_MTS_TRANSACTION results in a transacted operation if you make the call from within a COM+ transaction.

As in the case of an internal transaction, messages in a COM+ transaction aren't actually sent until the transaction has been committed. When you receive messages in a transaction, they're removed from the queue right away. If you call SetAbort to roll back a transaction, your messages are written back to the queue. Remember that there are no guarantees that the message will be placed back in the same position. The previous discussion of MSMQ isolation levels applies to external transactions as well as internal transactions.

Running a transaction with exactly-once semantics Now that you've seen how to pass transacted messages, let's examine the problems associated with running a COM+ transaction without MSMQ. COM and RPC are vulnerable to communication failures that can lead to inconsistent results. This is often unacceptable in an OLTP application. In such cases, MSMQ can provide a solution.

Let's look at what can go wrong when a client invokes a COM-based method call to run a COM+ transaction. There are three possible cases for failure. First, the method call's request message might fail between the client and the server. Second, the COM+ application might crash while the method call is executing a transaction. In both of these scenarios, the intended changes of the transaction aren't committed. The third failure scenario is when the method call's response message to the client fails or is lost after the transaction has been successfully run and committed.

So here's the problem. What if a client submits a transaction by invoking a COM-based method call on a remote COM+ object but doesn't get a response? The client application has no way of knowing whether the transaction has been committed. If the client submits the same request a second time, the transaction might be committed a second time. As you can see, this creates a problem that COM and RPC can't solve.

Transactional message queues provide a way to submit a transaction request with exactly-once delivery semantics. You can run a transaction with exactly-once semantics by breaking it down into three distinct phases in which messages are sent and received from transactional queues. Figure 10-5 shows a high-level picture of how the queues are set up.

Figure 10-5 Running a transaction request with exactly-once semantics involves using two transactional queues and three different transactions.

First, the client submits a message to a request queue. Then the listener application accomplishes three steps in a single high-level transaction: it receives the client's message from the request queue, runs the transaction requested by the client, and sends a receipt message back to the sender's response queue. The server must successfully accomplish all three steps or the high-level transaction will be rolled back. If the transaction is rolled back, the client's message is returned to the request queue. This means that the listener application can receive the message from the request queue a second time.

Once the requested transaction has been successfully committed, a corresponding receipt message is placed in the response queue. The client application then receives a transactional message from the response queue indicating whether the transaction was completed successfully.

Transactional queues allow the sender to determine which state a request is in: waiting to be processed, being processed, or already processed. If the sender sees that the original message is still in the request queue, the request hasn't been processed. If the sender finds a corresponding message in the response queue, it knows that the request has been processed. If there's no message in either queue, the sender knows that the request is currently being processed. RPC isn't as good as transactional message passing because it doesn't give the client the same ability to determine the exact status of a request.

Dealing with poison messages and exception handling When you design applications with transactional messaging, you have to watch out for a common design problem. This problem occurs when a listener application aborts the transaction in which it receives the client's request message. For example, a listener application might be required to abort a transaction due to a business rule violation or because the database server is off line. However, if the request message is simply written back to the request queue, it causes a recursive loop: the Listener application continues to receive the message, abort the transaction, and write the message back to the same request queue. This scenario is known as the poison message problem.

One common way to deal with poison messages is to run two separate transactions while processing the client's transaction request. The listener application creates an outer transaction to receive the request message, run a COM+ transaction to update the database, and send a response message back to the client. The COM+ transaction is run as a separate inner transaction. One easy way to accomplish this is to write a listener application that creates objects from a component whose transaction support property is set to Requires New.

Running two separate transactions can eliminate the poison message problem. If a business rule violation results in the rollback of the inner transaction, the listener application can determine that the client's transaction failed but still commit the outer transaction. This gives the listener application an opportunity to delete the original request message and send a response message with a failure notification back to the client.

Relatively speaking, it's not complicated to design a listener application that prevents poison messages that result from business rule violations, but you also have to deal with request messages that can't be processed due to other problems, such as system failure. When you design an application that will use transactional messaging, you should be careful not to underestimate the effort involved in dealing with poison messages and exception handling.

For example, what if a client submits a valid transaction request but the listener application can't process it right away? How do you write a listener application that can deal with a transacted request message when the database server is off line? You can't simply abort the transaction and write the request message back to the request queue. As you know, this creates a poison message scenario that puts the listener application into an infinite loop. Moreover, you can't simply commit the transaction and delete the request message without some type of contingency plan to prevent the client's request from disappearing into thin air. After all, the client submitted a valid transaction request and it's up to you to prevent it from falling on the floor.

One way to deal with this type of problem is to write an error message to a special exception-handling queue. For example, after receiving the transacted request message and determining that a system failure has occurred, you can send a copy of the message to an exception-handling queue that's regularly monitored by an administrator. This makes it possible for you to remove the original message from the request queue but still record the client's request. Once the database server is back on line, the administrator can run a utility program that you've written to process the request messages that have accumulated in your exception-handling queue.

I've presented just one simple approach to solving the exception-handling problems associated with transactional messaging. Many messaging applications have far more sophisticated designs. However, this example gives you an idea of the most important issues and how much time and effort is involved in creating a transactional listener application that does what it's supposed to do.