Connectivity

The heart of any application integration technology stack is the ability to extract information and invoke remote services contained in remote information systems one or many. This is the fundamental jumping-off point for application integration. The value of this type of technology is the ability to connect to complex (and perhaps archaic) systems and manage the production of information and services out of those systems, and at the same time, publish information to those systems as needed.

We can define connectivity as any pattern of technology, typically software systems, that connects into source or target systems through some type of point of integration (see Figure 20.1).

Figure 20.1. Connectivity.

We need connectivity for several reasons:

• First, we need some sort of reusable set of software services that can extract and publish information to source or target systems, which saves us from having to build those interfaces as "one-offs" each and every time we do an application integration project.

• Second, we need common interfaces into source or target systems that provide a consistent set of services.

• Finally, we need management visibility into connections to source and target systems' connections. We need this for reliability, as source and target systems are known to go down from time to time, so we can manage through those outages.

Support for Information-Oriented Connections

When dealing with simple information, information-oriented connectivity merely leverages whatever interface is available to extract information from the source system and move that information into the integration server. For example, customer address information coming from SAP may be transformed to account for differences in application semantics, and then published to a target system (or systems), say, PeopleSoft, using the same type of information-oriented connectivity.

Information-oriented connections offer certain advantages.

• First, we're dealing with simple information, so we usually don't have to change source and target systems. Most applications, certainly databases, already know how to produce and consume simple information.

• Second, we don't need to manage complex issues such as state, logic, and sequencing because there is no notion of behavior, just information.

• Finally, this approach is easy to understand and is in wide use.

In many cases, the information-oriented approach is the correct solution. Using service or business process integration to integrate systems is contraindicated for many problem domains when looking at the business problems they are trying to solve. In fact, you'll find that service and business process integration approaches to application integration are overapplied.

Accessing information within databases and applications is a relatively easy task, accomplished with few if any significant changes to the application logic or database structure. This is a tremendous asset because altering applications is not possible in many problem domains, such as supply chains, where you are likely dealing with systems that are beyond your direct control.

However, the straightforward appearance of information-oriented connectivity should not create the impression that it is simple. It is not. Migrating data from one system to another sounds straightforward and reasonable enough, but in order for information-oriented connectivity to actually work, architects and developers need to understand all integrated systems in detail.

Application semantics make this problem even more complex. Typically, application semantics in one system are not compatible with other systems the semantics are so different that the two systems just can't understand each other. For example, sales accounting practices might be different, as well as invoice numbers and customer sales data. Thus, Information-Oriented Application Integration is not just about moving information between data stores, but also managing the differences in schema and content (see the "Transformation" section later in this manifesto).

Support for Service-Oriented Connections (a.k.a. Web Services)

Service-oriented connectivity is a bit more complex, but nonetheless important. Instead of dealing with the extraction and publication of simple information to source or target systems, service-oriented adapters have to abstract services or application behavior as well. In other words, they need to expose application functions in a way that they may be abstracted into a composite application as a local function that actually exists on a remote system (see Figure 20.2). Remember, even though the function appears local to the composite application, the application processing occurs in the remote system that is connected through a service-oriented connectivity subsystem (adapter).

Figure 20.2. A composite application.

In essence, service-oriented connections take remote function calls within remote systems for other applications, and do so without having the applications understand anything about each other. Of course, service-oriented connections have to be linked through some type of integration server or application server to facilitate the extension into the composite applications, and to account for the differences in the applications, including semantics and enabling technology.

Service-oriented connectivity is a result of the interest in service-oriented architectures and the use of standards such as Web services, and perhaps some lingering use of more traditional distributed objects. However, creating service-oriented connectivity, in practice, is a bit of a challenge considering that the "connection" has to interact with internal application functions, rather than just application information. You're extending the behavior of that application to other applications, making the application functions appear native.

Service-oriented connectivity allows enterprises to share common application services as well as information. Enterprises accomplish this sharing either by defining application services they can share, and therefore integrate, or by providing the infrastructure for such application service sharing. Application services can be shared by hosting them on a central server, or by accessing them interapplication (e.g., through distributed objects or Web services).

Support for Coupling

Coupling, in the context of application integration, is the binding of applications together in such a way that they are dependent upon each other, sharing the same methods, interfaces, and, perhaps, data.

In essence, coupling is the tight binding of one application domain to the next. As a consequence of this requirement, all coupled applications and databases will have to be extensively changed to couple them. Further, as events and circumstances evolve over time, any change to any source or target system demands a corresponding change to the coupled systems, as well. Coupling creates one application and database out of many, with each tightly dependent upon the other. Service-Oriented Application Integration clearly leverages coupling in how applications are bound together.

Support for Cohesion

Cohesion is the "act or state of sticking together," or "the logical agreement." Cohesively integrated applications and databases are independent from one another. Changes to any source or target system should not directly affect the others. In this scenario, information can be shared between databases and applications without worrying about changes to applications or databases, leveraging some type of loosely coupled middleware layer to move information between applications and make adjustments for differences in application semantics.

You need to consider the tradeoffs. Cohesion provides the greatest flexibility as the application integration solution moves into the future, but it is not always the right fit. Some problems domains (e.g., service-oriented solutions) are better addressed with coupling.

Systems can be added to, changed, or removed from a cohesive application integration solution without typically requiring changes to any other system in the problem domain. Integration brokers provide the technology infrastructure of most cohesive application integration solutions. They are able to account for the differences between systems, accommodating differences in application semantics within a middle-tier process.

Despite cohesion's flexibility, if common business processes are to be reused, then a coupled approach provides more value. Distributed objects, transaction processing monitors, application servers, and, of course, Web services provide a good technology solution for a coupled application integration solution.

Support for Transaction-Oriented Connections

Transactional connections require that complex application requests be divided into bite-sized units called transactions. Transactional middleware, a component of an integration server or an application integration solution set (integration server, application server, or a combination), controls transactions from their beginning to their end, from the client to the resource server and then back again. It's important that we deal with transactions at the connection level because the remote functions contained in source or target systems must participate in the transaction.

In these scenarios, transactions are either all or nothing. Either they work or they do not. A transaction is never left incomplete. As a result, transactional middleware always leaves the system in a stable state. This provides the application integration architect with a reliable integration model to support transactional business solutions.

Support for Abstraction

The notion of abstraction, in terms of connectivity, lets us think about collections of data or services as abstract entities, thus represented in a form that is most useful to the integration server or the application integration architect. It's this notion that provides for the grouping of related pieces of information independent of their physical location and structure, as well as defining and understanding what meaningful operations can be performed on the data or services.

In other words, abstraction in its use with connectivity and application integration allows us to redefine the look, feel, and behavior of certain data sets and application services that are local to source or target systems. This means we can represent the data set or application service in a manner that best fits the needs of the application integration problem you are attempting to solve.

This also means enforcing certain restrictions on the use of the data or service, based on the needs of the application integration scenario, as well as streamlining the task of reasoning about the data or service. We've employed this concept with great success in the world of object-oriented technology for years.

What's more, we need to separate the implementation from the abstraction itself, so we can change the internal representation and/or implementation without changing the abstract behavior, and so people can use the abstraction in terms of connectivity without needing to understand the internal implementation.^[1]

^[1] Goldman, Kenneth J. 1996-97. "Data Abstraction."