Distributed Mediator Strategy

A more scalable approach to automating business processes compared to the Indirect Database Access strategy is to create a service that processes incoming and outgoing database requests through messaging middleware.

Scope

The complexity of the enterprise landscape requires a high degree of scalability, availability, and reliability. A .NET e-commerce engine may have to submit a purchase order for asynchronous and reliable database processing. A Java EE content management system may need to submit the product catalog changes to the database in an asynchronous manner and register with the database to receive certain event notifications.

Solution

The Distributed Mediator would ensure reliable coordination across distributed applications. Java EE and .NET applications can submit their requests to the Mediator that guarantees messages are delivered only once and arrive in order. This is achieved by using a message queue. The Mediator service incorporates logic to process the queue and ensure data integrity. A traditional business transaction is comprised of multiple operations. If one operation succeeds while another fails, the entire transaction has to roll back. Managing distributed transactions across asynchronous operations is the key functionality of the Distributed Mediator. Its job is to commit any changes upon successful execution or to issue a compensating transaction in case one of the operations fails. The Distributed Mediator service also addresses security constraints that are important to the enterprise by authorizing queue requests prior to their execution.

In the Microsoft world, the idea of a service that extends database functionality with support for long-running business processes and required systemic qualities is referred to as the Service Oriented Database Architecture (SODA), [SODA]. In a nutshell, SODA expands the concept of SOA to encompass the database interactions. A few commercial database management systems, such as SQL Server and Oracle, are extended with a message queue and the corresponding queue processing functionality. Oracle 9i Advanced Queuing, [OracleAQ], is an example of such a solution. Enterprise applications can produce (enqueue) or consume (dequeue) messages via the Oracle AQ component. Applications can communicate with AQ over JMS, HTTP, or e-mail. In addition to asynchronous reliable request processing, applications can register with AQ to receive event notifications. This can be achieved using OCI, PL/SQL, or IDAP APIs. Notifications can be optionally posted in the XML format in the form of an OCI callback, a PL/SQL procedure, an e-mail, or an HTTP post. To integrate with mainframe applications, AQ can propagate messages to the IBM MQ Series. AQ also supports integration with Tibco Rendezvous. Microsoft SQL Server 2005 offers a Service Broker component that processes incoming and outgoing database requests by using a message queue. Asynchronous guaranteed messages can be sent to the Service Broker queue by using extensions to Transact-SQL DML. Service Broker locking mechanisms ensure that individual messages related to the same task are processed only once. This feature allows other instances of the Service Broker to continue processing the remaining messages in the queue. Microsoft Service Broker supports XML-based storing and queue, important to implementing SOA. See [Intro_SQLServer2005_ServiceBroker] for more details. Figure 10-3 depicts this configuration where a Java EE or .NET application communicates with the Distributed Mediator component to asynchronously execute traditional CRUD operations.

Figure 10-3. Distributed Mediator strategy

Coordinating communication between two applications and a database is not trivial but is a manageable task. Incorporating a multitude of applications into a single business process becomes quite complicated. For elaborate business processes it may be necessary to separate the workflow engine to coordinate business transactions across a distributed application environment. Commercial workflow engines can be of great value, as they feature secure reliable integration points across disparate applications, BizTalk, SeeBeyond, and other commercial integration solutions, already mentioned in Chapter 6, "Resource Tier Synchronous Integration."

Benefits and Limitations

The main advantage of these strategies is that Java EE, .NET, and other enterprise applications can remain loosely coupled and safely share Resource tier components in an asynchronous manner. Of course, managing transactions and security context across multiple back-end applications could propose a challenge. A transaction that is part of the asynchronous workflow may result in an inconsistent data state. Therefore, with asynchronous processing it is always important to define a compensating transaction to roll back data into its original state. From an availability and reliability standpoint, the intermediate data queue should always be available. If an intermediate file or table is not accessible, this may become a bottleneck. If a message queue is used to store queries, the queue may become a single point of failure. Therefore, it is critical to determine a fail over strategy.

Related Patterns

The Gof [GoF] Mediator pattern is derived from the Distributed Mediator strategy. The purpose of the Distributed Mediator is to promote loose coupling between an enterprise application and the back-end system such as a database. It encapsulates details of asynchronous, reliable, secure processing.

Example

Previous chapters have discussed Java and .NET technologies corresponding to the database and message queue's interaction. These APIs include JDBC and JMS as well as .NET ADO.NET and MSMQ programming APIs. If Oracle or MS SQL Server 2005 are used, the Distributed Mediator functionality is provided out of the box. To manually implement the Distributed Mediator with a database management system like MySQL, the design outlined in Figure 10-4 should be considered:

Figure 10-4. Java EE-based Distributed Mediator service

The Java EE Distributed Mediator includes an EJB 2.0-compliant Message Driven Bean (MDB) that processes the message queue. The MDB would dispatch corresponding CRUD operations to the Data Access Object (DAO) that directly talks to the underlying database. The Distributed Mediator can be also used to pass the event notifications back to the Java EE applications. To see some sample code on how to develop asynchronous queries in Java EE by using a JMS provider, please refer to [AsyncQueriesJ2EE].

A similar design would take place on the .NET side, an example of which is detailed in Figure 10-5:

Figure 10-5. .NET-based Distributed Mediator service

In the figure, the MSMQ Façade component is responsible for the message queue processing. The Façade connects to the Data Access Layer that encapsulates a service to execute CRUD operations against the database.