The Future of Application Integration and Integration Servers

Integration servers, at least the core engine of integration servers, are destined to become commodity products because many perform only rudimentary functions. In an effort to differentiate their products, many integration server vendors are putting layer upon layer of value-added software on top of the core integration server engine. Thin adapters are giving way to thick adapters. Vendors are learning that it is better to hide the complexities of enterprise systems behind abstraction layers and put a business face on all the " geeky " little details. These abstraction layers save the end user from having to deal with the complexities of the technology and put the end user in a better position to take care of the business at hand.

Given this reality, when all is said and done, integration servers may turn out to be more about business integration than middleware.

Chapter Ten. Adapters and the J2EE Connector Architecture

There is no question about it: If you're going to do application integration, you're going to need adapters. That's the purpose of this chapter—to explain the notion, architecture, and implementation of adapters.

However, adapters don't stop there. There is a new standard on the rise, the J2EE Connector Architecture, which defines a standard architecture and adapter behavior. This standard gives us the ability to create an adapter once, and then use it anywhere —including most integration and application servers. Clearly this provides the application integration technology consumer with many advantages.

Adapters are powerful; they deal with the ugly details of the underlying application, database, and standards-based ( EDI , XML , etc.) interfaces. Believe me, you don't want to have to code through those interfaces on your own. It is always better to buy than to build.

The liberation of information and application services out of back-end systems is the job of adapters. Adapters sit between the source or target applications and the integration server, and account for the differences in the back-end systems by translating requests into something the native source or target application can understand, then they translate the response.

We need adapters for several reasons:

First, we need some sort of reusable set of software services that can extract and publish information to source or target systems, which saves us from having to build those interfaces as "one-offs" each and every time we do an application integration project.
Second, we need common interfaces into source or target systems that provide a consistent set of services.
Finally, we need management visibility into source and target systems' connections. We need this for reliability, as source and target systems are known to go down from time to time, and adapters can manage through those outages.

The simple notion of adapters becomes complicated by numerous vendor varieties, and even the differences between standards. Truth be told, the approaches to adapters outnumber the source or target systems they are looking to connect. Moreover, the use of standards, which is supposed to simplify things, may add more complexity and confusion in the short term . We'll talk more about that below.

The Purpose of Adapters

In the past, when we sought to connect Application A to Application B, or Applications A and B to an integration server, we had to interact with that application using some sort of interface that the application (hopefully) provided. Interfaces into applications and databases vary greatly, including everything from simple well-defined APIs to complex and cryptic exit calls, or perhaps something one level above screen scraping.

After hand-coding the interfaces to the middleware layer a few hundred times, in essence creating mini-applications between the source or target applications and the middleware layer, middleware vendors began to pitch the notion of adapters (see Figure 10.1).

Figure 10.1. Before adapters, developers had to code the connections between the application interfaces and the middleware.

graphics/10fig01.gif

Adapters are layers between the integration server and the source or target application. For example, an adapter could be a set of "libraries" that map the differences between two distinct interfaces—the integration server interface and the native interface of the source or target application—and hide the complexities of those interfaces from the end user or even from the application integration developer using the integration server.

An integration server vendor may have adapters for several different source and target applications (such as SAP R/3, Baan, and PeopleSoft), or for certain types of databases (such as Oracle, Sybase, or DB2), or even for specific brands and types of middleware. Over time, adapters have become more sophisticated. They began as a set of libraries that developers had to manipulate and evolved into binary interfaces that require little—if any—programming. Adapters are also getting smarter , with intelligence placed within the adapter, and with adapters running at the source and target systems in order to better capture events.

Two types of adapters exist in the context of integration servers: thin adapters and thick adapters. These adapters may have two types of behavior: dynamic and static.

Thin Adapters

Today's most popular integration servers offer thin adapters. In most cases, they are simply API wrappers, or binders, which map the interface of the source or target system to a common interface supported by the integration server (see Figure 10.2). In other words, they simply perform an API-binding trick, binding one API to another.

Figure 10.2. Thin adapters don't provide sophisticated layers of software between the source or target systems and the integration server, and are really just simple abstractions on top of existing APIs.

graphics/10fig02.gif

Thin adapters have the advantage of being simple to implement. With no additional, "thick" layer of software between source and target applications, there is greater granular control. Thin adapters have a number of disadvantages, however. Because using them accomplishes nothing more than trading one interface for another, thin adapters impact performance without increasing functionality. And a fair amount of programming is still required. To complicate matters, the common APIs that are being mapped are almost always proprietary.

Thick Adapters

Unlike thin adapters, thick adapters provide a significant amount of software and functionality between the integration server infrastructure and the source or target applications. The thick adapter's layer of abstraction makes managing movement or invoking processes painless (see Figure 10.3). Because the abstraction layer and the manager negotiate the differences between all the applications requiring integration, almost no programming is needed.

Figure 10.3. Thick adapters place a sophisticated abstraction layer on top of the application interfaces.

graphics/10fig03.gif

Thick interfaces accomplish this via the layer of sophisticated software that hides the complexities of the source and target application interfaces from the integration server user.The user sees only a businesslike representation of the process and the metadata information as managed by the abstraction layer and the adapter. In many cases, the user connects many systems through this abstraction layer and the graphical user interface, without ever having to resort to hand-coding.

Repositories are major players in the thick-adapter scenario. As we noted, the repository understands much of the information about the source and target applications and can use that information as a mechanism to interact with the source and target applications on behalf of the integration server.

In addition, several abstraction layers may be created around the types of applications to be integrated. For example, there may be an abstraction for common middleware services (such as distributed objects, message-oriented middleware, and transactional middleware). There may also be an abstraction layer for packaged applications, and another layer to address the integration of relational and nonrelational databases. This structure hides from the end user the complexities of the interfaces that each entity (middleware, packaged applications, and databases) employs.

Another feature of a thick adapter is the ability to think and process information on its own, independent of the integration server. Processing performed by thick adapters may include

Simple transformation of application semantics, such as the transformation of a native format to a canonical form.
Exception-handling routines.
Information queuing and restart in case of integration server outages.
Direct routing of information that may not need service from the integration server.

With the many advantages and conveniences of thick adapters, it should come as no surprise that integration server vendors are moving toward them. Their progress is slowed by the fact that thick adapters require a tremendous amount of time to develop, as much as six times that of a thin adapter. Right now, this time investment deters some vendors. However, as application integration becomes more sophisticated, enterprises will continue to look for more sophisticated solutions that require no programming and provide an easy, businesslike method to view the integration of the enterprise.

Static and Dynamic Adapters

As noted earlier, in addition to being thick or thin, adapters are also defined by being either static or dynamic.

Static adapters , the most common adapters in play at this time, must be manually coded with the contents of the source and target systems. They have no mechanism for understanding anything about the schema of connected databases. As a result, they must be configured by hand (i.e., coded) to receive information from the source schema. If the connected database schema changes, static adapters have no mechanism to update their configuration with the new schema.

In contrast to static adapters, dynamic adapters "learn" about the source or target systems connected to them through a discovery process that takes place when they are first connected to the application or data source (see Figure 10.4). This discovery process typically means reading the database schema information from the repository (or perhaps the source code) to determine the structure, content, and application semantics of the connected system. More important, dynamic adapters not only learn about the connected system, they also are able to relearn information if something changes within the connected system over time. A dynamic adapter automatically understands when a customer-number attribute name changes.

Figure 10.4. When using dynamic adapters, information such as schema becomes visible to the integration server.

graphics/10fig04.gif

Type Importers

To address the dynamic nature of source and target systems, many adapters and their connected integration servers provide type importers, mechanisms that read schema information from the source or target systems and automatically define it as an input or an output. As source and target systems change over time, schema information can be reimported as needed.

The use of type importers is important because it saves the end user from having to define by hand—and ultimately risk mistakes—the input and output schema for connected applications.

Type importers can be passive, meaning you have to invoke the initial importing or the reimporting of the schema from the source or target systems. Or they can be active, meaning that importing and reimporting schema (when changed) occurs automatically.

Centralized versus Distributed Adapters

Two distinct adapter architectures are emerging: distributed and centralized. As the name suggests, centralized adapters run with the integration server. Generally, these are thin adapters that only bind the integration server's API to the API of the source or target application.

Just as centralized adapters are thin adapters, distributed adapters are thick adapters that exist on the integration server as well as on the source or target application. Running an adapter on the application that is being integrated allows many processes of the source or target application to be better determined, such as capturing events, monitoring states, or even restarting the application as required. And because the adapter is in two parts , it is better able to coordinate the transfer of information between the integration server and the source or target application.