ACID Transactions

The field of transaction processing is by no means a new discipline. Transaction processing infrastructures such as CICS and Tuxedo, OTS/JTS and MS DTC have been around for decades, and much innovative and interesting work continues in the field today. However, underlying the development of transaction technology over the decades has been one pervasive notion: ACID.

ACID is the acronym used to describe the classical form of all-or-nothing transactions, which are Atomic, Consistent, Isolated, and Durable. These properties are of such fundamental importance to transaction processing systems that they warrant further elaboration, even in a Web services book. The ACID characteristics are:

• Atomic The transaction executes to the end, or logically appears to not have executed at all. This is particularly important when executing business logic (such as the bank account debit-credit) that involves the updating of multiple underlying data sources, where the atomicity property turns a set of operations into a single indivisible logical operation.

• Consistent Any data that has been updated during the lifetime of the transaction is left in a consistent state at the end of the transaction (provided it was, of course, consistent to begin with), irrespective of any failures that have occurred during that transaction. This usually means that database primary keys remain unique and the database has referential integrity (everything that a record refers to actually exists), or that the conditions set by certain business rules are met (for instance, ensuring an overdraft status is less than the total permissible overdraft that an account will grant).

• Isolated Any running transaction believes that it has exclusive access to the resources associated with it. The classical case for the isolation property stems from a situation where several debtors each chase the last $50 from an account. If the two transactions to transfer money from the debtor's account to the creditor's are allowed to run un-isolated, then they will both claim the remainder of the money, though clearly it is impossible for them to withdraw $100 when there is only $50 available! Thus for the isolation property to be met, the outcome of running the transactions must be equivalent to if the individual transactions had run one at a time. Ensuring isolation is usually achieved through locking of shared data. In this case, the balance of a bank account would be locked to prevent concurrent access and maintain isolation.

• Durable A durable transaction system ensures that even in the event of failure, the work of the transaction will not be lost, but will instead be persisted on some form of storage (like disk or a database) that will typically survive any failures. Durability is usually implemented by the transaction engine writing an intentions log to persistent storage throughout the work of the transaction, and by carrying through the results of its intentions to the durable data source once the transaction is successfully brought to an end. Conversely, if the transaction fails for some reason, then the intentions log can be discarded (ensuring that none of the operations logically ever occurred) or used to replay the transaction up to its point of failure and re-try the partially finished work.

Having established the fundamental properties of ACID transactions, it is worthwhile taking a look at how such abstractions are used within today's enterprise before we move up to running transactions at the Web services level. As we discussed earlier, in the simplest form, a transaction is a logically undoable unit of work. For illustrative purposes, we shall return to the bank account example since it is well known and relatively straightforward to comprehend. It is important to remember that we're dealing with principles in this section, and that industrial-strength transaction processing systems have vastly more sophisticated implementations than the mechanisms we're going to examine. However, the principles remain valid and with that caveat in mind we can examine the simple debit/credit transaction shown in Figure 7-1.

Figure 7-1. A successful debit/credit transaction.

Figure 7-1 shows the classical transaction example where one bank account is debited and another is credited. In this example there are no failures while the work is being undertaken, and the flow of work through the transaction is therefore straightforward.

1. The transaction is started and the database records that are involved in the transaction are locked (so that the isolation property is maintained).

2. The transaction then enters its work phase where a check is made to see whether there are sufficient funds available to continue with the transfer. In this case, we can assume that such funds do exist. (Where funds aren't available, the transaction may finish now and unlock the records early, freeing them up for use by other transactions).

3. The intention to withdraw funds from the debtor's account is recorded with a shadow write (a write that updates a temporary copy of the actual data) but, importantly, the changes to the debtor's account are not yet made the changes are in the show copy only.

4. The intention to credit the creditor's account is also recorded with a shadow write, though again the actual account is not updated.

5. The transaction reaches its termination phase where the intended outcome for the transaction (i.e., to commit or abort to make durable or not the effects of the shadow writes) is synchronously recorded in an intention log, which then becomes the definitive source for finding the outcome of the transaction.

6. As the intention is to commit the transaction, the effects of the shadow writes are committed to the actual records in persistent storage.

7. Once the termination mechanism is satisfied that the effects of the intentions log have indeed been committed to persistent storage (e.g., through strictly synchronous disk writes), then the transaction can terminate and the intentions log can be discarded and locks released.

In a typical transaction we operate on copies of the actual online data and only when we are satisfied that the work has come to a successful conclusion do we allow the copies to replace that original data. While successful transactions are interesting enough, it is far more entertaining to look at what happens when a system failure causes the transaction itself to fail, and show how such system failures can be masked by transactions. In the example shown in Figure 7-2, a system failure causes some difficulty for the executing transaction with the worst possible timing the debtor's account has been provisionally debited, while no monies have yet made their way to the creditor's account.

Figure 7-2. An unsuccessful debit/credit transaction.

Assuming the system that supports the transaction shown in Figure 7-2 can be re-started, the system must then recover any transactions that were in progress prior to the crash. In our example case, this is relatively straightforward since it is a matter of aborting the transaction simply by discarding shadow writes and unlocking any resources (database records in our case) that were associated with the transaction, as shown in Figure 7-3. This has the effect of completely reversing the transaction with no side affects, other than the fact that processing the payment will have been delayed by the amount of time that the system takes to rerun the work.

Figure 7-3. Recovering an unsuccessful debit/credit transaction.

Failure during the termination phase presents a different problem, but one which can be solved nonetheless. Imagine a situation such as that depicted in Figure 7-4, where the transaction has successfully progressed to termination, but where a system failure has caused the transaction to fail before the termination phase has completed.

Figure 7-4. A transaction failing during the termination phase.

In the situation portrayed in Figure 7-4, once the system has started its recovery process, it is a relatively simple matter to read the intention log and to effectively replay the termination phase to completion. Since no records have yet been unlocked, the data remains inaccessible to other transactions, which although it may cause inconvenience is perfectly safe. Using the decisions recorded in the intentions log, with the stored shadow writes recorded during the work phase, the recovery process can simply rerun the procedure, as we can see in Figure 7-5.

Figure 7-5. Recovery from a failed transaction during the termination phase.

Though the implementations are admittedly naïve, the concepts we've examined here are generic enough to provide a basic understanding of what a transaction is and how it provides its dependable unit-of-work abstraction. However, the examples we've considered have been centralized the transactions have executed with a single program and operated on locally available records. This is clearly not the ideal architecture on which to base Web services transactions, but fortunately there is a precedent and known techniques for running ACID transactions in a distributed environment.