Why Is Messaging Important?

[Previous] [Next]

The basic idea behind messaging is simple. A message is a request or other type of notification that's sent from one application to another. A queue is a named, ordered repository of messages that lives on a specific computer. Some applications issue requests by sending messages to the queue, while other applications remove messages from the queue and carry out the processing requested by the sender. Figure 10-1 shows how a queue is typically used in a distributed application.

A sender application creates and prepares a message by setting various properties in the message header and packing parameterized information into a payload, which is called the message body. It then writes the message to the queue. A receiver application removes the message from the queue as it reads it so the message is processed only once. The receiver application interprets the request, unpacks parameterized information from the body, and runs any commands requested by the message sender.

Figure 10-1 A sender application creates and prepares a message and then writes it to the queue. A receiver application removes the message from the queue, interprets the message's request, and executes whatever processing the message requires.

Message queues also support the concept of a reader application, which can examine a message in a queue without removing it. The act of reading a message is known as peeking. Peeking allows an application to be selective about the messages it removes from a queue.

Any application can assume more than one of these roles. It's common for an application to act as both a reader and a receiver on a single queue. The application can look at what's in the queue by peeking and receive only messages that meet certain criteria. It's also common for an application to receive messages from one queue while sending messages to a second queue. For example, it's common to create a workflow application in which various programs have both in-box queues and out-box queues. The out-box queue for one program is the in-box queue for another. After a program processes a request, it simply forwards the request to the next program in the workflow.

A messaging subsystem is a valuable building block in a distributed system because it allows applications to submit requests and send information to other applications in a connectionless, asynchronous manner.

Messaging Versus RPC and HTTP

In some ways, message passing is similar to a synchronous protocol such as RPC or HTTP; in other ways, it's very different. As you've seen throughout this book, COM uses RPC to issue interprocess requests between clients and objects. In Chapter 9, I explained how computers in a Web-based application communicate using HTTP. It's important to understand that RPC and HTTP are synchronous protocols. Their synchronous nature creates some noteworthy limitations, which I'd like to take some time to describe.

Let's start by looking at a couple of common problems with synchronous communication that you can solve by using messaging. The first problem with a synchronous protocol such as RPC or HTTP is that a client application's thread blocks during each request. The client application must wait while the server computer performs the requested task and returns a response. Also, if the server computer has a large backlog of requests from other clients, a client application might block on a response for an unacceptable period of time.

A messaging protocol, on the other hand, doesn't force client applications to wait. A client application can continue its work immediately after submitting an asynchronous request. You can return control back to the client faster, so you can improve response times. The trade-off is that you can't immediately inform the client of what happens because the message isn't processed until after the client regains control. The main assumption with messaging is that something must get done, but it doesn't have to be done right away. It's acceptable for the server to process the request in a few seconds, an hour, or a few days.

The second problem with a synchronous protocol is that it requires an established connection between two specific computers. In the case of a simple request sent from a client computer to a server computer, both the client application and the server application must be on line at the same time. For example, if the server is off line, a client application can't submit requests. Likewise, if no client applications are on line, the server application can't process any requests; it just sits around with nothing to do. In essence, neither side can get any work done unless the other side is up and running.

The requirements of connection-oriented communication are even higher in systems where requests are routed across two or more server computers. Figure 10-2 depicts a physical deployment scenario in which a client requests flows across three server computers.

Figure 10-2 Applications that rely exclusively on synchronous, connection-oriented protocols don't exhibit very high levels of fault tolerance. Every server computer must be on line in order for the system as a whole to be operational.

The client sends a request to the Web server using HTTP. This request, in turn, fires an ASP page on the Web server, which uses DCOM and RPC to invoke a method on a business object running on an application server. The application server establishes an ADO connection to SQL Server using the native OLE-DB provider, which uses a proprietary, synchronous protocol called Tabular Data Streams (TDS). The important thing to note is that all three server computers must be on line and operational in order for the client to submit a request and receive a successful response. If any of the server computers goes off line, the application as a whole experiences downtime.

As you can see, distributed applications that rely exclusively on synchronous protocols can't provide the highest levels of fault tolerance and availability. An application server or a database management system (DBMS) might go off line due to a system crash or scheduled maintenance.

A messaging subsystem can offer higher levels of fault tolerance and availability because it acts as a buffer between the application that's issuing a request and the application that's processing the request. Client applications can continue to send request messages to a queue regardless of whether the receiver application is on line. When the server application comes back on line, it can resume processing request messages that have accumulated. A server can also continue its work after client applications have gone off line. As you can see, the queue acts as a buffering mechanism that allows either side to accomplish its work in the absence of the other.

The example in Figure 10-3 is similar to that in Figure 10-2, with one exception: Communication between the Web server and the application server is conducted using MSMQ messages instead of DCOM and RPC. The use of an asynchronous protocol means that clients can continue to make requests when either the application server or the database server has gone off line. The infrastructure of MSMQ simply stores request messages on the Web server until they can be successfully forwarded to the application server. The application server or the database server can go off line without causing any downtime for the overall application.

Figure 10-3 Messaging offers higher levels of fault tolerance because clients can continue to successfully issue requests when one or more server computers is off line.

A third problem with using a synchronous protocol is that requests are typically processed on a first come, first served basis. There's usually no way to prioritize calls with RPC or HTTP. A high-priority request must wait its turn if low-priority requests were submitted ahead of it. A messaging subsystem such as MSMQ can solve this problem by assigning priority levels to messages. Messages with a higher priority level are placed at the head of the queue, while lower-priority messages are placed at the end. The server can thus respond to the most important messages first.

The fourth (and arguably the most significant) problem with RPC and HTTP is that they're vulnerable to failures that lead to inconsistent results in OLTP applications. For example, let's look at what can go wrong when a client application invokes an RPC-based method call to run a COM+ transaction. Three types of failure can happen. First, the method call's RPC request message might fail between the client and the server. Second, the COM+ application might crash while the method call is executing a transaction. In both of these scenarios, the intended changes of the transaction are not committed. The third failure scenario is that the method call's RPC response message to the client might fail or be lost after the method call has successfully run and committed the transaction.

What happens if a client executes a method on a transactional object using DCOM and RPC from across the network and it doesn't receive a response. There's no real way of determining what happened. The client can't simply try to execute the same method because there's a chance the transaction will be run twice. From this example, you should observe that DCOM and RPC can't offer exactly-once delivery guarantees. HTTP is vulnerable to the same problems.

Transacted messaging provides a way to submit a transaction request with exactly-once delivery semantics. You can submit a transaction request with exactly-once semantics by breaking it down into three smaller transactions. In the first transaction, the client application sends a request message to a request queue on the server. In the second transaction, a listener application receives the request message, processes the request, and sends a response message to the client's response queue. In the final transaction, the client application receives the response message to find out what happened.

The important point here is that RPC is inferior to message passing because it doesn't allow the client to determine the exact status of a request. Later in this chapter, after I address the basics of MSMQ programming, I'll revisit this point and explain how to set up an application that allows you to run transactions with exactly-once semantics.

Now that you know how queues can improve your distributed applications, let's put this knowledge to work using MSMQ. As you program with MSMQ, you'll see that it requires more effort than writing COM-based method calls. When you use DCOM and RPC, you basically get all of your interprocess communication for free. When you use MSMQ, you have to invest more time in both application design and programming. However, when you consider the limitations of RPC, MSMQ is often well worth the time and energy you invest.