Chapter 7: Coordinating Transactions in Web Services

In today s enterprise-scale applications, it s crucial to be able to coordinate multiple resources, each potentially running on various servers or even on different platforms. As you know, one of the major benefits of XML-based Web services is their ability to enable interoperability in such heterogeneous and distributed enterprise environments. In addition to interoperability, enterprise applications also require a standards-based method for enabling Web services to engage in coordinated business processes and transactions and to expose existing enterprise resources. This chapter covers the basics of coordination in enterprise applications and discusses how this notion of coordinated processes can be extended into the realm of Web services. You ll discover how two proposed specifications, WS-AtomicTransaction and WS-Coordination, can be used to provide coordination for Web services.

Introducing Transactions

Before diving into a discussion of coordination in Web services, I want to spend a little time reviewing the basics of transactions, particularly distributed ones, and describing how Microsoft has implemented this functionality in the past. A transaction is a set of operations involving a diverse set of resources that can be managed as a single logical operation. A transaction should exhibit the following properties (known as ACID properties):

• Atomicity Either all operations in the transaction are executed or none of them are.

• Consistency When the transaction is completed, all data must be left in a consistent and expected state.

• Isolation Any operation in the transaction should not be modifying data that is also being modified by an operation in another transaction or data that is in an intermediate state. In other words, the effect on the data should be the same as if each transaction were running alone.

• Durability Changes made by the transactions are persisted in a recoverable state, such as writing to disk.

Transactions that occur within the scope of a single data source or service are known as local transactions; and transactions involving multiple data sources and services that must be accessed using standard networking protocols are known as distributed transactions.

Local Transactions

Confined within the scope of a single data source or service, a local transaction consists of a set of data changes (create, read, update, delete data modification language (DML) operations affectionately known collectively as CRUD), all of which must be completed successfully to maintain the consistency of the data. For example, consider a Web service that places sales orders. In this scenario, each new sale completed by the service requires that a new row be inserted into a sales order table and one or more related rows be inserted into a sales item table. In this case, the rows inserted in the sales item table represent individual items in the order, which is represented by the row in the sales order table. Because of this relationship, these rows must all be inserted together; otherwise , sales items might exist without a corresponding order or an order might exist without any items in it. Should a failure occur during processing, we might end up in such a state, with orphaned data rows. To prevent this from happening, all of the insert operations should be executed within a single transaction. That way, if any of the inserts fail, the entire transaction can be rolled back and the data can be maintained in a consistent state.

For local transactions, all of the required resources are managed by the resource manager for the local data source or service, sometimes called the transaction manager. However, things get more complicated for transactions comprising operations from multiple data sources or services. These transactions must be coordinated among two or more individual resource managers.

Distributed Transactions

Today most transactions that require the coordination of multiple resources occur in distributed transactions. These transactions involve a group of operations on a set of heterogeneous data sources, each of which is managed by its own resource managers and which are distributed across either a local or a wide area network. A possible distributed transaction might involve a SQL Server Database and a Microsoft Exchange Server. If changes in a database have to be applied in coordination with a change to a user s mailbox, the resource managers of these heterogeneous data sources must be able to coordinate their efforts. To facilitate transactions in such environments and maintain the ACID properties of the transaction, a single process needs to be responsible for coordinating the entire transaction. This process is known as the distributed transaction coordinator (DTC).

Distributed Transaction Coordinator

The DTC is responsible for enrolling resources in a distributed transaction by negotiating with the resource manager for each resource that s participating in the transaction. The DTC is also responsible for obtaining the approval from each participating resource manager before completing a transaction, as well as for notifying each about the final outcome of the transaction. Figure 7-1 represents a distributed transaction involving three separate data sources on two separate servers and the DTC running on a third server. The DTC communicates remotely with the three resource managers using a standard networking protocol, in this case TCP. The dotted line represents the logical boundary of the transaction.

Figure 7-1: Representation of a distributed transaction

Distributed transactions in Microsoft Windows are supported by the Microsoft Distributed Transaction Coordinator (MSDTC). MSDTC supports Microsoft COM+, which is in turn used by .NET Enterprise Services, which provides a managed access to MSDTC. Rather than use an interoperable Web service, MSDTC employs both the OLE Transactions interface protocol and the X/ Open Distributed Transaction Processing (DTP) model to enable interaction with a variety of resource managers, including relational databases, object- oriented databases, file systems, document storage systems, and message queues. X/Open DTP XA is a DTC standard supported by managers of non- Microsoft resources such as Oracle, IBM DB2, Informix, and Sybase. As I will soon discuss, the WS-Coordination and WS-AtomicTransaction define a SOAP- based alternative to these tightly- coupled , object-based distributed transaction protocols.

Two-Phase Commit Protocol

To guarantee the ACID properties of a distributed transaction, the DTC implements a two-phase commit process. Because of its ability to maintain ACID properties of the transaction, this type of transactional coordination process is the workhorse of distributed transactions. As you might expect given its name , a two-phase commit process has two phases. In the first, or prepare , phase, the DTC for the transaction asks each individual resource manager participating in the transaction to vote on whether the transaction can be completed. When a resource manager votes to commit the transaction, the requested resources are locked and placed in a recoverable state, such as written to disk, so that if a crash occurs during the commit phase the transaction can be recovered. At this point, the resource is prepared to commit.

The second, or commit , phase occurs if and when all of the resource managers have confirmed that their transactions are in a state where they can be committed. At that point, the DTC sends out a commit request to all of the resource managers telling them to complete their transactions and release the resources. The DTC is then notified by each resource manager when its transaction has been committed. When any resource fails in the prepare phase, the DTC issues a rollback command, telling each resource manager to undo the changes made in the prepare phase.

To make this clearer, consider a hypothetical interaction between a sales order application and an inventory application. When a new order is placed, the sales order application makes a request to a separate inventory management application to verify that sufficient inventory exists for a new order to be placed. If sufficient inventory exists, then the inventory application withholds the requested amount of inventory from its stock. In addition, new order information is created in the sales order application. Such a scenario requires the use of an atomic transaction protocol to prevent updates from being committed in one application without the related updates being committed at the other. If not, a failure might occur during processing, which would result in items being removed from inventory without the order being placed or an order being placed without the items being withheld from inventory. I will discuss this type of example scenario in the context of the coordination specifications later in this chapter.

Long-Running Business Processes

While the two-phase commit process helps to ensure the atomicity of coordinated distributed transactions, the major drawback of a two-phase commit operation is that it locks valuable resources during the transaction. This is not usually a problem when the resources in a transaction are able to respond in a timely manner. However, because of the emergence and growth of business-to-business (B2B) electronic commerce taking place over the Internet, a new type of distributed transaction has emerged. Unlike traditional distributed transactions, coordinated business processes, also called business orchestrations, often rely on request messages that might not be responded to in minutes, hours, or even days.

Consider the following scenario: A customer sends an e-mail query to a vendor for a particular item. After the vendor checks the inventory, a response message is sent back to the customer saying that the item is in stock. The customer sends back a message to place the order, and this message contains a purchase order number. At this point, the vendor makes a request to the automated distribution system to retrieve the order from the warehouse and ship it to the customer. At the same time, the vendor sends an electronic invoice to the customer. Within 30 days of receiving the invoice for the order, the customer wires an electronic payment to the vendor, who, after receiving it, sends a payment confirmation message back to the customer.

Even when fully automated, this coordinated business process will take at least a month to complete; however, it still arguably represents a single logical operation. It s clear from the foregoing example that this type of process can in no way be atomic since you would never lock resources for such an extended period of time. These types of coordinated business processes actually comprise many individual atomic transactions, which have their own transaction coordinators. While not strictly adhering to ACID properties, coordinated business processes still need to behave predictably. In fact, the failure model of a coordinated business process is much more complex than in an atomic transaction. In a coordinated business process, each transaction is completed as if the entire process were going to succeed; this leaves much more cleanup work to be done and more logic to be worked through should a failure occur somewhere in the process.