6.3 Persistence and Transactions in Queues

Queues can be configured as persistent or transient. A persistent queue is stored on disk and is therefore protected against system failure. A transient queue is not protected against failure, but because it requires no disk access, it can run faster. I assume that persistent queues will be the norm for drawbridges. Drawbridges are a lot like databases; we almost always trade performance for reliability.

Queues can also be configured as transactional or nontransactional. A transactional queue acts as a transactional resource. A nontransactional queue does not. I talked about transactional resources back in Chapter 3 (Transactions), but let me give a brief refresher here. A transactional resource is a system that knows how to converse with DTC (the distributed transaction coordinator ), and thereby participate in what, in Chapter 3, I called a tightly coupled multiple-resource transaction.

A tightly coupled multiple-resource transaction has special characteristics when one of the transactional resources is a database acting as a data strongbox and the other is a message queue acting as a drawbridge. Consider a business component inside the donor fortress that first updates its data strongbox and then makes a work request of another fortress. The following sequence occurs:

1. The business component starts a new transaction. This transaction is intercepted by DTC (as discussed in Chapter 3), who goes into her meditative phase.

2. The business component updates the strongbox. DTC notices the update.

3. The business component makes an envoy request. The transaction flows to the envoy, meaning that any work the envoy does will be done within the same transaction as the work done by the business component.

4. The envoy creates an infogram and places it in the drawbridge (message queue). DTC notices that a message has been placed in the queue.

5. The envoy receives confirmation from the message queue that the message has been received.

6. The envoy returns control to the business component.

7. The business component commits the transaction, letting DTC know that the end transaction boundary has been reached. DTC enters the consensus phase.

At the end of this sequence, DTC is ready to coordinate the transaction between two transactionally aware resources: the database (representing the data strongbox) and the message queue (representing the drawbridge).

DTC now asks both the database and the message queue (all of the transactional resources that she noticed when she was in the meditative state) whether they are willing to accept the updates that each of them has received during this transaction.

When DTC gets an answer from the last resource (either the database or the message queue; we don't know the order in which she will ask), she switches to the final state. If both the database and the message queue were willing to "commit" to commit, then in the final state, she tells them to go ahead and do the commit. If either is unwilling to commit to commit, she tells them both to forget it.

All transactionally aware resources know how to process the Commit request. This is part of the requirement of being a transactionally aware resource. However, the resources can decide for themselves what it means to commit. The database interprets a Commit request as a request to process the transaction in an all-or-nothing fashion.

The message queue interprets the Commit request differently. If the message queue is asked to insert a message, then commit means accepting the message and " guaranteeing " delivery. As I mentioned earlier, however, the message queue can't really guarantee delivery. All that it can guarantee is that it will hold the message for delivery until somebody (namely, the recipient guard) gets around to reading the message.

Suppose DTC decides that she can't safely commit the transactionfor example, because the donor strongbox, in a grumpy mood, refuses to commit to commit. What does she do with the message queue? She says to forget it. "Forget it," in formal lingo, is known as a rollback.

When a message queue is told to roll back, it ejects the message. This means that the drawbridge is returned to the state it was in before the infogram was inserted. If the recipient guard were to check for a message after a rollback, it would find none.

If the message queue is asked to deliver a message, as would be the case on the recipient fortress side, then commit means "giving up" the message and rollback means taking the message back into the message queue. When a message queue rolls back, it is as if the guard never read the message in the first place. The next time the guard method loop executes, it will find the very same message waiting patiently.

An odd transactional relationship exists between the workloads in the donor and the recipient fortresses . Each fortress works within its own independent transaction, but the two transactions are related . The first (donor) transaction coordinates the work done by the donor fortress with the placement of the infogram into the drawbridge. The second (recipient) transaction coordinates the receipt of the infogram by the recipient fortress with the work that the recipient fortress needs to do to fulfill that request. These relationships are shown in Figure 6.2.

Figure 6.2. Donor and Recipient Transactional Relationships

The donor and recipient transactions are related to each other in two ways. The first relationship is one of sequence. The recipient transaction cannot begin until the donor transaction has committed because the infogram is not really placed into the message queue until then.

The second relationship is one of a loose guarantee. The fact that the drawbridge is a transactional resource more or less guarantees that either both the donor and the recipient transactions will commit or neither of them will. What is somewhat remarkable is that this coordination occurs without DTC. She is needed to coordinate each transaction individually, but she is not needed to coordinate the two together.

To see how this works, let's consider the possibility that the donor transaction fails. We won't worry about a business failure (e.g., insufficient balance to make the withdrawal), but a failure because either the message queue (drawbridge) or the database (strongbox) declines to make the commitment to commit. In this case, both the drawbridge and the strongbox will be told by DTC to roll back. If this happens, the message will be ejected and not delivered. When the recipient guard looks for a message, it won't be there, and nothing will happen in the recipient fortress. So both transactions will effectively failthe first because it is rolled back, and the second because it never starts.

Now let's consider the possibility that the donor transaction succeeds but the recipient transaction fails, again because either its strongbox or its drawbridge declines to commit to commit. In this case, both the recipient strongbox and the drawbridge will roll back. When the drawbridge rolls back, the message will be sucked back into the message queue.

At this point the donor transaction has committed (it must have, or the recipient guard never would have seen the infogram), but the recipient transaction has rolled back. Notice that even in this scenario, we have a loose guarantee of transaction coordination between the donor and recipient fortresses because even though the recipient has rolled back, the infogram is still in the drawbridge. The next time the guard checks, it will still be there. If the recipient transaction fails again, then once again the infogram is sucked back into the drawbridge. In theory, sooner or later whatever is causing the problem (probably the strongbox, since strongboxes cause most transactional problems in this world) will relent and the transaction will succeed.

In practice, the recipient transaction may still fail. Failure could result for two reasons. First, if the recipient transaction failed repeatedly, sooner or later the message queue underneath the drawbridge would lose interest in the message and move the erstwhile infogram to a limbo no-man's-land. The second way the transaction could fail is that the recipient fortress could, in its business logic, decide to discard the request and still commit the transaction.

In this case the drawbridge would consider its duty to deliver the infogram as having been fulfilled. So the coupling of the donor and recipient transactions is loose in the sense that the system will do its best to accomplish both or neither transaction, but it can't make an absolute guarantee in the same way that DTC can when she has full control of all the transactional resources. However, the coupling is still much tighter than most people realize.

In any case, DTC is not an option for coordination of the donor and recipient transactions. Remember, DTC requires the holding of database locks and the flowing of transactions across drawbridges. As I discussed in Chapter 5, we don't flow transactions across drawbridges, regardless of the functionality that the vendors provide.

In light of this background on asynchronous drawbridges, let's look more closely now at the heterogeneous and homogeneous varieties.