Queued Components

[Previous] [Next]

The Queued Components (QC) service that's part of COM+ and Windows 2000 can provide many of the benefits of MSMQ programming, but in a much more transparent manner. QC integrates COM and MSMQ to provide the simplicity of COM-style method calls along with the benefits of messaging and MSMQ. As you've seen throughout this chapter, MSMQ programming requires more attention than COM does when it comes to issuing and responding to client requests.

When you use MSMQ, you have to think much more about the mechanics of communication. You have to prepare messages and explicitly send them to a specific queue across the network. On the receiving side, you must write a listener application to harvest these MSMQ messages and process their associated requests. Once you add in the extra complexity of establishing bidirectional communication (including response queues and Correlation IDs, for example), it's obvious that MSMQ programming is much more tedious than submitting requests using COM-based method calls.

One of COM's greatest strengths is its ability to abstract away the gory details of RPC and remote communication. As you saw in Chapter 3, the COM runtime and the universal marshaler work together to create a remote connection by transparently inserting a proxy/stub layer between a client and an object. This proxy/stub layer provides location transparency because it can seamlessly remote method calls between the client and the object.

In one sense, remote communication with MSMQ is similar to remote communication with COM. Someone sends messages from one computer and someone else listens for these messages on another computer. But COM is much simpler than MSMQ. COM's standard proxy/stub layer takes care of preparing, transmitting, and receiving RPC messages behind the scenes. With MSMQ, you have to do most of that work yourself.

A queued component is a COM component. But a queued component's methods can be remoted from one machine to another using MSMQ as an underlying transport, which means that queued components don't suffer from the same RPC-related limitations as standard COM components. As you'll recall, RPC imposes quite a few limitations because of its connection-oriented, synchronous nature.

Using a queued component is relatively simple. Here's how things work at a very high level. An application on the client computer creates a special QC proxy object and invokes one or more method calls. The COM+ runtime remotes these method calls by recording them in an MSMQ message and sending the message to a system-provided queue on the server computer. QC provides the required plumbing of a built-in listener service on the server computer. This listener service allows the COM+ runtime to monitor the queue and harvest messages as they arrive. After harvesting a message, COM+ creates an instance of the queued component and replays the messages that were recorded on the client computer.

You can see that QC provides the benefits of messaging while abstracting away the details of MSMQ programming. In the next section, we'll look into the finer details of using QC. You should note that QC doesn't provide as much flexibility as programming directly with MSMQ. If you understand the tradeoffs between using QC and using MSMQ, you can select the best approach for the project at hand.

The Architecture of Queued Components

Let's begin by revisiting the architecture of COM's standard proxy/stub layer, as shown in Figure 10-6. This architecture allows you to establish an RPC connection between a client and an object. As you'll recall from Chapter 3, when a client invokes a method, the proxy prepares and sends an RPC request message to the stub. After executing the method on the object, the stub prepares and sends an RPC response message back to the proxy. The details of preparing, sending, and receiving RPC messages are hidden behind the scenes by the proxy and the stub.

Figure 10-6 COM's remoting architecture requires that a proxy/stub layer be introduced between the client and the object. The proxy and the stub rely on RPC to conduct synchronous, connection-oriented communication.

Figure 10-7 shows the architecture of QC. When the client application properly instantiates an object from a queued component, it's not bound to the object that contains the method implementations but rather to a special proxy object called the recorder. In this sense, queued components have an architecture that's similar to COM's standard proxy/stub layer. A client can't tell the difference between a COM object, a standard COM proxy, and a QC recorder. However, the implementation details of the recorder are much different from that of a standard COM proxy.

Figure 10-7 A queued component can use MSMQ as its underlying transport. On the client side, method calls are automatically recorded and sent by the QC recorder component. On the server side, the QC listener service calls upon the QC player component to replay calls recorded in the MSMQ message.

While a standard COM proxy knows how to communicate with the stub using RPC, the QC recorder object forwards method calls using MSMQ. The COM+ runtime creates recorder objects on the client computer from a configured component named QC.Recorder. The standard installation of COM+ adds this system-supplied component to a COM+ library application named COM+ Utilities.

Let me point out an important difference between a standard COM proxy and the QC recorder. COM's standard proxy/stub layer requires an established RPC connection. Each COM method call that's issued by the client is forwarded to the object in real time. For example, if a client executes two methods, the first call must return to the client before the second call can be executed. Queued components, on the other hand, are designed to work when the client computer can't establish a connection to the server computer.

When a client application creates an instance of the QC recorder and starts executing methods, the calls are not forwarded in real time. Instead, a recorder object buffers all the calls on the client computer by recording them in a single MSMQ message. The QC runtime doesn't try to send the message to the server computer until after the recorder object has been released. It's important to note that QC relies on the transactional messaging facilities of MSMQ. The fact that each message is sent inside the scope of an MSMQ transaction means that communication with QC has exactly-once delivery semantics.

I'll cover the details of configuring the server computer in a bit. Let's first look at the bigger picture. On the server computer, the queued component must be installed in a COM+ server application that's been configured as queued. When you configure a COM+ server application to be queued, COM+ automatically creates an application-specific public input queue for it. (COM+ also creates a few additional exception-handling queues.) Once a server application has been configured as queued, it can also be configured as a listener. When you configure a server application as a listener, COM+ sets up a service to monitor the queue and respond to incoming messages.

Let's assume that a queued component and its hosting COM+ server application have been properly configured and that the application is running. Now let's say a client application creates a recorder object and calls three methods. The recorder object responds by serializing the data required to replay these three methods in the body of an MSMQ message. When the client application releases the recorder object, the QC runtime attempts to send the message to the application's input queue on the server computer. If the client computer is disconnected from the network, the local queue manager stores the message and forwards it later, after it's been reconnected to the network. When the message is finally delivered to the application's input queue on the server computer, the QC listening service sees that it has arrived and begins to process it.

The QC listening mechanism involves a nonconfigured component named QC.Listener. While the listener monitors the queue, it delegates the responsibility of processing each MSMQ message to a player object. The listener creates a new player object for each incoming MSMQ message. Note that the listener creates player objects from a system-supplied configured component named QC.ListenerHelper. As it does with QC.Recorder, the standard installation of COM+ adds the QC.ListenerHelper component to the COM+ library application named COM+ Utilities.

A player object creates an instance of the queued component and executes each method recorded on the client computer. The MSMQ message carries with it all the information that the player needs to do its job. Once the player object has replayed all of its methods and released the instance of the queued component, its job is done. It is then released by the COM+ runtime.

Let's quickly review what it takes to remote a series of method calls using a queued component. Each series of method calls requires quite a few things. It requires the creation and destruction of a recorder object on the client computer, and it requires the COM+ runtime on the client computer to send one MSMQ message to the server computer. On the server computer, it requires the listener service to create and release a player object. Finally, during the player object's lifetime, the player object is responsible for reading information from the MSMQ message, creating an instance of the requested queued component, and replaying each method before releasing the object.

I'd like to point out a few key aspects of this architecture. First, player objects are thread-neutral. They run in the COM+ server application's MTA thread pool. If you're writing your code with Visual Basic, objects created from your queued components are loaded into threads in the STA thread pool. The bad news is that there's a thread switch between a player object and a Visual Basic object. The good news is that this architecture distributes all of your objects across all the threads in the STA thread pool. This means that QC provides multithreaded behavior. Your application can process several MSMQ messages concurrently, which leads to higher levels of throughput than what you can achieve with a single-threaded listener application.

Another point to keep in mind is that both the recorder component and the player component are configured to require a transaction. This has important implications for the server computer. The listener starts a new COM+ transaction when it creates a new player object. If you configure the transaction support attribute of a queued component as either Supported or Required, each instance of the queued component is created in the player's transaction. This is significant because aborting the player's transaction prevents QC from successfully processing a message received from the application's input queue. I'll revisit this topic a little later when I talk about exception handling.

Limitations of Queued Components

Now that I've covered the high-level architecture of QC, I can describe a few of its notable limitations. The first limitation is that QC relies on the MSMQ runtime. MSMQ must be installed on every client computer and server computer that uses QC.

The second important limitation is that QC's architecture relies on configured components on both the client computer and the server computer. The listening plumbing on the server also requires the COM+ container application DLLHOST.EXE. This means that QC can be used only by applications that run on a Windows 2000 computer. This unfortunate limitation prevents many companies from using QC for client-to-server communication. If your users are running earlier versions of Windows, QC might provide only a means for achieving asynchronous, disconnected communication from one server computer to another. For example, you might be able to use QC only for sending requests from a Web server to an application server.

The other noteworthy limitations involve several miscellaneous things you can do with MSMQ that are inaccessible through QC. Unlike MSMQ, QC doesn't allow you to query Active Directory for a queue using criteria such as a queue label. QC doesn't allow you to peek at messages. QC doesn't allow you to use Correlation IDs to correlate response messages. For one application, QC might provide all the functionality you need. For another application, you might decide to use MSMQ because it provides some needed feature that's not supported by QC.

Designing Queued Components

You can create a queued component in the same way that you create other configured components—by adding a multiuse class to an ActiveX DLL project. A queued component must implement at least one interface that's queueable. You should note that it's possible to create a user-defined interface that's queueable with either Visual Basic or IDL. Once you define a queueable interface, you can implement it in a multiuse class. However, in this chapter I'll keep things simple. I'll show you how to create a queued component by using a multiuse class with public methods. I've chosen not to work with a separate user-defined interface because I want to show the easiest way to create a queued component using Visual Basic.

A queueable interface has one important constraint: It can be used to communicate in only one direction. Therefore, the interface implemented by a queued component can contain methods with only input parameters. In other words, parameters can be sent only from the client computer to the server computer. Unlike COM method calls, which rely on RPC-based request/response pairs, a queued component can't return parameters from an object back to its client. Here are a few examples of methods that can be defined in a queueable interface:

 Public Sub Method1()     ' Implementation End Sub  Public Sub Method2(ByVal i as Integer)     ' Implementation End Sub  Property Let Property1(ByVal s As String)     ' Implementation End Property 

As you can see, every parameter must be marked using the ByVal attribute. An interface isn't queueable if it contains one or more methods that require an output parameter or a return value. For example, let's say you're trying to create a queued component using a multiuse class with public methods. The public methods that make up the default interface must all be queueable. Any of the following method signatures would render the class unusable as a queued component:

 Sub Method3(ByRef i As Integer)     ' Implementation End Sub ' ByRef is the default calling convention. Sub Method4(i As Integer)     ' Implementation End Sub Function Method5() As Integer     ' Implementation End Function Property Get Property1() As Integer     ' Implementation End Property 

The key to designing queued components is to avoid method signatures that require passing data from the object back to the client. If you write the implementation for a multiuse class that follows these rules and compile it into a component in an ActiveX DLL, you can install it in a COM+ server application and configure it as a queued component.

Configuring a Queued Component

The first thing you do to set up the server computer is to create and properly configure a COM+ server application. Let's say you create a COM+ server application named MyApplication. You can manually configure this application to be queued and to be a listener by using the Component Services administrative tool, as shown in Figure 10-8. You can also automate the creation and configuration of a COM+ server application using scripts or using the COM+ Admin objects.

When you configure a COM+ server application as queued, COM+ automatically creates a public transactional queue with the same name. COM+ also creates several private queues to assist QC with exception handling.

Figure 10-8 You can configure a COM+ server application as queued and as a listener on the Queuing tab of the application's Properties dialog box in the Component Services administrative tool.

You must do two more things to set up a queued application. First, you must configure the server application as a listener so that COM+ will monitor the application's queue for incoming messages. Second, you must start the application. The listening service works only when the application is running. You can start the server application manually using the Component Services administrative tool or you can start it programmatically using the COM+ Admin objects. Here's an example of Visual Basic code that starts a COM+ server application:

 Dim cat As COMAdminCatalog Set cat = New COMAdmin.COMAdminCatalog cat.StartApplication "MyApplication" 

You can also start a COM+ server application with a Microsoft Windows Script Host (WSH) batch file written in VBScript. Then you can configure the server computer to run this batch file on system startup by using the Windows Scheduler service. You can also schedule a queued application to run in the middle of the night if you'd like to process client requests in batch mode as opposed to real time.

In addition to configuring the application, you must also configure each queued component. First, you install the component in the queued server application. Then you configure the interface to be queued. Let's say that you want to use the default interface of a component named MyQueuedComponent. Once the component has been installed, you can locate the default interface (which is named _MyQueuedComponent) in the Interfaces folder of the Component Services administrative tool, as shown in Figure 10-9.

Figure 10-9 Each configured component contains an Interface folder.

Once you locate the interface, you can invoke its Properties dialog box. On the Queuing tab, you can configure the interface to be queued, as shown in Figure 10-10. If the Queued check box is disabled, it means COM+ has determined that the interface isn't queueable. This happens when a method in the interface has a ByRef parameter or a return value. Note that configuring an interface to be queued, like most other aspects of COM+ administration, can be done programmatically as well.

Figure 10-10 An interface for a queued component must be configured as queued to enable asynchronous communication.

Let me quickly summarize the configuration requirements for the server computer. First, you need to create a COM+ server application that's configured as queued and is configured to be a listener. Second, you must install the queued component in the application and make sure that one or more interfaces is configured as queued. The order of these two steps doesn't matter. Finally, you must start the server application. If you configure a server application to be a listener while it's running, you have to shut it down and restart it in order for the change to take effect.

Programming Queued Components from the Client

The most important aspect of client-side programming is creating the recorder object properly. If you want queued components to use MSMQ as an underlying transport, a client application must create a QC recorder instead of a standard COM proxy.

Let's say you've properly configured the queued component and its hosting COM+ server application, you've used the COM+ Export command to create an application proxy for this application, and you've installed the application proxy on the client computer. At this point, you're ready to use queued components from the client computer.

Note that the client-side configuration required for queued components is similar to that for other COM components. The client computer must have a registered copy of the type library as well as configuration information for ProgIDs, CLSIDs, and IIDs. It's also helpful to have a client-side AppID with a valid RemoteServerName. All of these details are handled for you if you install an application proxy on the client computer.

When you create a QC recorder object to act as a proxy for a queued component, you can't instantiate it using a standard technique such as calling the New operator or the CreateObject function. Look at the following code:

 Dim obj1 As MyDll.MyQueuedComponent Set obj1 = New MyDll.MyQueuedComponent Dim obj2 As MyDll.MyQueuedComponent Set obj2 = CreateObject("MyDll.MyQueuedComponent") 

Both of these activation techniques result in standard COM activation. An instance of the queued component is created on the server computer, and the client is connected to it through COM's standard proxy/stub layer. Each method call is executed against the object in real time using RPC. In this scenario, you have a queued component but you're using it like a standard COM component.

To properly create the recorder object on the client computer, you must use a special activation technique. Look at the following code:

 Dim rec As MyDll.MyQueuedComponent Set rec = GetObject("Queue:/New:MyDll.MyQueuedComponent") 

As you can see, you can properly create the recorder with a call to Visual Basic's GetObject function. However, in order for you to understand why this works, I need to explain the string argument that's passed to GetObject and explain the concept of a moniker.

Monikers have been part of COM for some time. A moniker is an object that finds or creates another object and binds it to a client. A moniker can be identified with a string such as the one passed to the GetObject function. I don't say much about monikers in this book because you can't implement them with Visual Basic. But you can use preexisting monikers from a client application written in Visual Basic using a call to GetObject.

The COM+ team added two new monikers to Windows 2000 to accommodate the needs of QC: the Queue moniker and the New moniker. When you call GetObject and pass a string that identifies the Queue moniker, a special moniker object is created behind the scenes, which creates a QC recorder object and binds it to the client. The Queue moniker relies on the New moniker to do its job. In the preceding example, I created a Queue moniker string in this format:

 Queue:/New:MyDll.MyQueuedComponent 

I passed the queued component's ProgID to the New moniker. You can also pass a CLSID with or without braces. This means there are three different formats that work the same way.

Note that the Queue moniker can accept parameters that make it possible to open the queue and send messages in a customized manner. You can see a list of all the available options in the Queued Components documentation in the Platform SDK. Here's an example of using a parameter to specify a server name that will override the declarative remote server name setting:

 Queue:ComputerName=MyServer/New:MyDll.MyQueuedComponent 

Now let's discuss the life cycle of the recorder object. While the recorder object remains active, method calls are simply recorded and buffered on the client computer. The QC runtime doesn't attempt to send the MSMQ message across the network until you release the recorder object. Look at the following code:

 Dim rec As MyDll.MyQueuedComponent Set rec = GetObject("Queue:/New:MyDll.MyQueuedComponent") rec.Method1 rec.Method2 33 rec.Prop1 = "Some Value" Set rec = Nothing 

It's not until the last line, when the client releases the reference to the recorder object, that the QC runtime attempts to send the message. And, of course, if the client computer can't establish a connection with the server computer or another routing server in the enterprise, MSMQ caches the message locally until it can be forwarded to its proper destination.

The QC documentation also points out that you can create objects from a queued component using the New moniker, without the Queue moniker. Here's an example:

 Dim obj As MyDll.MyQueuedComponent Set obj = GetObject("New:MyDll.MyQueuedComponent") 

Instantiating an object from a queued component in this manner has the same effect as the techniques I showed you earlier that use the New operator or the CreateObject function. The New moniker without the Queue moniker results in a call to CoCreateInstance. This means the client doesn't create a recorder object. Instead, it directly creates a remote instance of a queued component and is connected across a standard proxy/stub pair. This technique results in synchronous RPC-based communication. It also requires an established connection between the client computer and the server computer. If a direct connection can't be established between these two computers, the activation request fails.

It was a design goal of the COM+ team to create an infrastructure that made it easy to switch back and forth between synchronous and asynchronous calls. As you can see, once you've properly set up a queued component you can access it through connection-oriented, synchronous communication (COM) or through disconnected, asynchronous communication (QC). You can easily switch back and forth between these two radically different styles by simply changing the string the client application passes to GetObject.

Bidirectional communications

Establishing bidirectional communication with QC is possible, but it can also be tricky. The support that QC provides for returning response messages to a client isn't overly intuitive, especially when compared to how easy it is to use QC for sending messages in one direction. Let me provide an example to demonstrate what's required. The client application, in addition to creating a recorder object for the queued component on the server, must create a second recorder object using a local queued component. The client must send this recorder object as a parameter in an outgoing method. The queued component on the server can then use the recorder object that's been passed to return a series of method calls to the client computer.

It definitely adds complexity to an application when you want to use QC to send response messages back to client applications. You must create and configure a queued component and a queued COM+ server application on the client computer as well as the server computer. This approach works best when the client application is also another COM+ server application. However, it's not very elegant for a client application with a user interface.

What's hard about setting up a callback from a client application with a user interface? The problem is that the client application runs in one process while the callback objects are created and run in a separate instance of DLLHOST.EXE. It's tricky because you have to coordinate communication between these two processes.

Queued Components and Exception Handling

Let's step through how QC works when everything goes according to plan. The recorder object on the client computer sends an MSMQ message to the server computer. Remember that this message is sent to a transactional input queue. The listener service then creates a player object to receive the message. The QC player component is configured to require a transaction. Therefore, a player object runs in a COM+ transaction and can make a transacted receive. If the player object can successfully create an instance of the target queued component and successfully replay all the methods, it gives its consent to commit the transaction. When the player commits the transaction, QC assumes that the client's request has been processed, and the MSMQ message is deleted.

But what happens if things don't work out so smoothly? The infrastructure of QC provides support for exception handling and preventing poison messages. As I mentioned earlier, when you mark a COM+ application as queued, COM+ creates six additional private queues for exception handling in addition to the application's primary input queue. COM+ creates these extra queues to avoid the poison message problem.

Think about what would happen if the player object simply aborted the transaction when experiencing an error during the replay of a method. The MSMQ messages would be returned to the primary input queue and the QC infrastructure would try to reprocess the same message over and over again in an infinite loop. QC provides the additional queues to deal with exception handling in a more elegant fashion.

A few things can prevent a player object from successfully processing a client's request and deleting the associated MSMQ message. First, a player object aborts the transaction if it experiences a runtime error while replaying the methods from the MSMQ message. Second, any object created in the same transaction as the player object can also abort the transaction. For example, if your queued component is configured to require or support transactions, its objects are created in the same transaction as a player object. If your object calls SetAbort, the transaction aborts and the MSMQ message isn't deleted. Finally, the transaction times out if a player object can't complete its work within the computer-wide transaction timeout interval. By default, this interval is 60 seconds. A player object must replay every method, release your object, and commit the transaction before the timeout interval expires. If it doesn't do all this, the COM+ runtime automatically aborts the transaction.

When the player object's transaction is aborted, QC doesn't simply return the message to the same queue. If it did, it would experience the poison message problem. Instead, QC moves the message to the next queue in a chain of retry queues. In addition to an application's primary input queue, COM+ creates five retry queues and a final resting queue.

If you create a queued application named MyApplication, COM+ creates a public input queue with the name myapplication. It creates five retry queues with the names myapplication_0, myapplication_1, myapplication_2, myapplication_3, and myapplication_4. It creates a final resting queue named myapplication_deadqueue to store messages that can't be processed.

Here's how it works. If a player object can't successfully process a message from the primary input queue, QC moves the message to myapplication_0. After 1 minute, QC tries to process the message three more times. If these attempts fail, QC moves the message to myapplication_1, waits 2 minutes, and then attempts to process the message three more times. If the message continues to cause problems, QC moves it across all five retry queues and then into the final resting queue.

Each exception queue has a longer wait time than the queue before it. The wait times escalate from 1 minute to 2 minutes to 4 minutes to 8 minutes and finally to 16 minutes. After waiting 16 minutes with the message in myapplication_4, QC makes three final attempts and then moves the message to the final resting queue, where it becomes your responsibility to deal with it.

COM+ also supplies a utility named MessageMover, which you can use to automate the moving of messages between queues. MessageMover is a system-supplied component that's part of the COM+ runtime (COMSVCS.DLL). You can program against MessageMover in a Visual Basic application by including a reference to the COM+ Services Type Library.

Let's look at an example that shows how this utility component might be useful. Let's say you come in Monday morning to find that your application's database server has been off line for the last 48 hours. If clients submitted requests using queued components over the weekend, the final resting queue could be full of MSMQ messages that couldn't be processed. Using MessageMover, you can easily create an administrative application to move all these messages from the final resting queue back to the primary input queue. Once the database server is back on line, you should be able to move all the messages and QC will be able to successfully process them.

Programming with the MessageMover is easy. The following code can be added to a Visual Basic application to move all the messages from the final resting queue back into the primary input queue.

 Dim MessageMover As MessageMover Dim MessageCount As Long Set MessageMover = New COMSVCSLib.MessageMover MessageMover.SourcePath = ".\private$\myapplication_deadqueue" MessageMover.DestPath = ".\myapplication" MessageCount = MessageMover.MoveMessages MsgBox MessageCount & " messages have been moved" 

You can also create an exception class that can be called upon to automate the handling of a message that's been moved to the final resting queue. You create an exception class by creating a component that implements an interface named IPlaybackControl from the COM+ Services Type Library. This interface has two methods, FinalServerRetry and FinalClientRetry. You can write an exception class component to deal with failed messages on the client side as well as the server side. An exception class must also implement the queued interface that the client used to record methods in the original MSMQ message.

You can associate an exception class with a queued component by using the Advanced tab of the component's Properties dialog box in the Component Services administrative tool. In the Queuing Exception Class text box, enter the ProgID or the CLSID of the exception class.

When QC moves a message to the final resting queue, it looks to see whether the queued component has an exception class. If it does, QC creates an object and calls QueryInterface to establish an IPlaybackControl connection. Then QC calls FinalServerRetry. The way you implement this is fairly open-ended. For example, you can use the MessageMover utility to move the message to another queue. You can also send an e-mail to an administrator or log a Windows event.

After calling FinalServerRetry on the exception class object, QC calls QueryInterface once more to obtain another connection to the interface that defines the methods recorded in the message. QC can then replay the methods that were originally recorded into MSMQ by the client. This scheme gives the author of an exception class the opportunity to write contingency code to deal with the fact that the client's methods could not be replayed successfully. Once again, you can implement these methods in a number of ways. Note that if the call to FinalServerRetry and the calls to the individual methods succeed, QC assumes that you've taken care of things, and the MSMQ message is deleted. If you raise an error in the exception class, the message is moved to the final resting queue.

As you can see, QC supplies a pretty elaborate framework for dealing with poison messages and exceptions. In one respect, this is helpful because creating such a framework yourself isn't a trivial undertaking. In another respect, this framework doesn't provide as much flexibility as you might need in certain situations. If your application requires handling of poison messages in a different manner, you might want to avoid QC altogether and resort to raw MSMQ programming. The trade-off with this approach is that you have to create your own listener application and hand-roll your own exception-handling framework.