Chapter 2: Defining Your Application Integration Environment | Microsoft Corporation

At a simple level, application integration means helping a series of applications communicate with one another better. To ensure that the integration is both beneficial and feasible, however, you should closely examine the business processes that must be integrated before focusing on the applications themselves. If you understand the business processes that your organization uses, you can map those processes to your application integration requirements. This chapter discusses the different levels at which application integration can occur, and shows which capabilities your application integration environment may require at each level. It also examines some of the issues you are likely to face when creating your application integration environment.

Levels of Application Integration

Application integration can occur at many different levels, including:

Presentation level
Business process level
Data level
Communications level

Presentation-level integration can be a very effective way of integrating applications in your environment. Presentation-level integration allows multiple applications to be presented as a single cohesive application, often using existing application user interfaces. This guide does not cover presentation-level integration in any detail.

The following paragraphs discuss the other forms of integration in more detail.

Business Process–Level Integration

Business process–level integration often forms the starting point for defining your application integration environment, because it is most closely related to the business requirements of your organization. Business process–level integration starts with defining your business processes and then specifying the logical integration that corresponds to those processes.

Defining Your Business Processes

Before you can design your application integration environment, it is important to understand your organization in logical terms, removed from the technology that underlies it. At this stage, you are not interested in whether information needs to travel from one computer to another using messaging, using file transfer, or using request/reply. Nor are you particularly interested in what protocols are used, or whether it is necessary to perform semantic mapping between two systems. Those considerations, and others, are necessary later, but first you need to define your business processes.

Business processes consist of a group of logical activities that model the activities in your organization. These activities typically represent the smallest units of logical work that can be done in your organization. A typical example of an activity is Update Inventory, which is invoked when new inventory arrives, or when a customer order is placed.

Business processes usually have a beginning and an ending state, and generally have action-oriented names. They perform work and can be measured while doing so. For example, the business process Process Order might represent a series of activities, including Enter Order, Accept Payment, Schedule Production, and Update Inventory. When the order is processed, it goes from an unprocessed state to a processed state. The order-processing rate can be measured to show the efficiency of the process.

Any applications that are related in a given business process are candidates for integration. The reason for integrating the applications is to help the business process in some way; by making it faster, less error-prone, or less expensive, for example.

As you define your business processes, you should consider the following factors:

Place. Can the business process be completed in one location, either for the client or the business? If a change of location is required, the business process cannot be completed in one time interval, and its state must be stored.
Time. Can the business process be completed in one interaction with the business, or does it need to wait for some events, meaning that it occurs over multiple time intervals? If it cannot be completed in one interval, for example due to legislation, then state must be stored.

These dimensions define the complexity of the business process. They also drive the complexity of the integration of IT services in terms of whether the IT service is available at that place and time, and whether the service has the necessary business process state information. The complexity of the business process can also be affected by the nature of the applications involved in integration; for example, their maturity, degree of integration, and so on.

Business Process Modeling

Business process modeling consists of defining the logical processes in your environment, identifying the start and end points, and determining the logical path between those points. The models can be generated in text or in flowchart form.

You should use business process modeling to define the details of each of the business processes that occur in your organization. Some of the steps may currently be automated; others may represent manual processes. The more complete your models, the easier it is to design an application integration environment that closely matches the needs of your organization.

Modeling Simple Business Processes

Some business processes are very simple. They consist of a single interaction with a business and are completed in a single time interval. Someone (for example, a customer or partner) or something (an automated request) contacts the business and makes a request. This request may be made through a member of staff, or through a self-service facility. For a typical simple business process, the organization has a process that can handle the request immediately, and the process closes. There is no need to save the state of the business process (although there may be a need to record the occurrence of event and the result). Examples of simple business processes include:

Registering to use services on a Web site
Requesting an inventory of goods of a particular type
Asking for a quotation for a service

Business processes can be quite complex. They can occur over multiple time intervals and in multiple places, and they can require state information about the business process to be managed. Often the steps include a combination of automated and manual steps. Figure 2.1, on the next page, shows a model of a sample complex business process.

click to expand
Figure 2.1: Model of a complex business process

Note

The following description does not match the key in Figure 2.1, because the numbers in the figure define order processing steps that humans perform, while the following steps explain the entire business process.

The business process shown in Figure 2.1 consists of the following steps.

Someone (for example, a customer or partner) or something (an automated request) contacts the business and initiates an order for goods either through a self-service mechanism or through an order-entry representative of the business.
The Order Management system captures the order details. The system recognizes that this customer has placed enough orders with the company to qualify for a discount on the goods. The system includes this fact with the order information that it has gathered. The system marks the order in the database as ‘order in process’.
The event originator is told that the order is under way and that progress updates will be coming.
A business staff member is aware of the order status (either because he or she is handling the order, or because he or she has received a notification from the external event handler IT service). This staff member initiates a request to check for inventory or delivery. This request is a person-to-person communication that may be initiated by e-mail, written note, or phone. Key details are passed to a staff member in the Inventory Management department.
The Inventory Management staff member checks that enough stock exists to fulfill the order. In this case, it does not, so the staff member issues a service request to a supplier for more stock and a promise of delivery date. This service request may be initiated by e-mail, paper form, or phone. The order details are not changed.
At some point, the supplier replies to Inventory Management. A staff member in the department updates the stock position in the supporting IT system. The staff member checks the orders waiting for inventory, finds the appropriate order, and then notifies the Order Management department that this order can now be fulfilled from stock on the date that the supplier has promised.
A staff member in Order Management updates the department database with this information, and issues a notice to the event originator confirming the order and the terms. Order Management passes the order information to the Billing department and asks Billing to issue the bill, using the new special terms.
Billing recalculates the bill, updates the billing database, and sends the bill to the customer. The bill notifies the customer that the goods will be shipped when payment has been received.
When Billing receives payment, a staff member updates the billing database and then notifies the Logistics department that the order can be shipped.
A staff member in Logistics schedules delivery and notifies the customer of when to expect the goods.

As this example suggests, a complex business process can be relatively difficult to model. Where you encounter problems in modeling a complex business process, you should break down any complex constituent processes into a number of simpler ones. Occasionally, however, the processes are so complex that it is impossible to model them accurately.

Business Processing Standards

One advantage of business processes being entirely logical is that they have no dependence on the underlying technology. This enables you to define business processes according to predefined standards, which has the following benefits:

You can interchange business processes more easily between organizations, with trading partners interacting at the process level rather than at the data level. This results in simpler business-to-business interaction.
You can recruit individuals who are already familiar with the standards you have adopted.
You can deploy software that has built-in support for those standards.

Existing business processing standards represent both the language and the visual representation of those processes. This allows you to define your business processes textually or visually, and then use tools to translate them into the corresponding language representation.

Business processes are most commonly represented by XML-based languages. The two most commonly used languages are Business Process Execution Language for Web Services (BPEL4WS) version 1.1 and Business Process Modeling Language (BPML) version 1.0.

The Business Process Modeling Notation (BPMN) version 1.0 specification, released by the Business Process Management Initiative (BPMI) Notation Working Group, allows both BPEL4WS 1.1 and BPML 1.0 to be represented using common elements. BPMN supports process management by providing a notation that is intuitive to business users and yet is able to represent complex process semantics.

Mapping Business Requirements to Your Application Integration Requirements

After you have determined the business processes that your organization uses, you can begin to examine how they affect your application integration requirements. It is important to realize that a simple business process does not always correspond to simple requirements for application integration. In fact, a simple business process may require complex interaction between multiple applications.

Figure 2.2 shows a simple business process that requires the integration of multiple applications.

click to expand
Figure 2.2: Simple business process requiring application integration

By modeling each of your business processes, you can define which applications must integrate with one another, and in what way. This enables you to make intelligent decisions about the specifics of your application integration environment.

Business Processing Integration Capabilities

Table 2.1 shows a number of capabilities that are required for effective business processing integration.

Table 2.1: Business Processing Capabilities
Capability	Description
Rules Processing	Allows you to define rules and process them at run time.
Business Transaction Management	Ensures that if a business activity fails, related activities are canceled.
Workflow	Controls the interaction between people and automated processes.
Orchestration	Coordinates the execution of the different activities that make up a business process.
State Management	Maintains the state in the context of pending applications.
Event Processing	Recognizes when business events have occurred, logs those events, and determines which business process capability receives the event.
Schedule	Generates events when a defined time period has elapsed or at a particular time of day. Also tracks processes in which an external event should occur within a specified time frame or by a specified date or time.
Contract Management	Monitors and processes contractual obligations at run time.

For more information about each of these capabilities, see the Appendix, "Application Integration Capabilities."

Data-Level Integration

You can define how applications communicate with each other at a business process level, but if they cannot understand the data they exchange, they cannot integrate successfully. Because different applications often handle data in different ways, a number of capabilities are required to establish integration at the data layer.

There are two general ways to enable data-level integration:

Add logic to enable each application to understand incoming data from other applications.
Add logic to enable each application to interpret outgoing data to an intermediate data format and interpret incoming data from that format into a form the application understands.

Note

In each case, the logic may modify the application itself or, more commonly, may place an interface in front of each application.

The problem with the first approach is common to most areas of application integration — a lack of scalability. In most enterprise situations, application integration architecture should instead adopt the second approach, with the data level using an intermediate data language.

Different integration scenarios have different data-level requirements. For example, if you synchronize data between two applications that store customer information, the data may need to be formatted to fit the data format of the target application. However, if the data is being moved into a data warehouse or data mart, the data may not only have to be formatted, but it may also have to be sorted, filtered, or otherwise processed so that it suits the needs of the users.

The nature of the applications you integrate may alter the way in which your data capabilities can function. For example, if you are moving data between applications in real time, the data-level capabilities generally work on one message at a time, and yet they may still need to support a very high volume of transactions per hour. To meet these needs, the data capabilities must normally be small, quick, and multithreaded. Other applications may only need to transfer data in batches (for example, loading a data warehouse every night at midnight). In this case, the data capabilities may not have to be multithreaded, but they may need to be particularly strong at sorting, summarizing, and filtering sets of data.

To support the various complex data integration requirements, your application integration solution must often contain a considerable amount of logic that supports the access, interpretation, and transformation of data. You also need schematic models of enterprise data to describe data attributes. The definition and recognition of schemas enables the validation of data. Descriptions of how data elements may be related or mapped to each other allow for the transformation of data.

Data Integration Capabilities

Table 2.2 shows a number of capabilities that are required for effective data integration.

Table 2.2: Data Integration Capabilities
Capability	Description
Data Transformation	Modifies the appearance or format of the data (without altering its content) so that applications can use it.
Data Validation	Determines whether data meets the predefined technical criteria.
Data Access	Determines how data is collected and presented to applications.
Schema Definition	Maintains predefined schemas.
Mapping	Manages the relationship between applications and determines how the data must be transformed from the source to the target application.
Schema Recognition	Checks that the schema can be read and understood, and that it is well-formed.

For more information about each of these capabilities, see the Appendix, "Application Integration Capabilities."

Communications-Level Integration

Although the forms of integration already discussed are very important, they all depend on integration at the communications level. Unless applications can communicate with each other, you cannot integrate them.

Not all applications are designed to communicate in the same way. Some communicate synchronously, others asynchronously. Some use file transfer to communicate; others interact through messaging or request/reply.

The nature of the applications you are integrating often determines how communications occur between them. Older applications may use file transfer for communications, and some may not be designed to communicate with other applications at all. Newer applications often use message-based communication and may use predefined standards to facilitate communication. Before determining the requirements for your communications-level integration, you need to examine the communication needs for your environment and the existing capabilities of your applications.

Enabling Communication Between Applications

Because many applications are not designed to communicate directly with each other, it is likely that you will need to do some development work to ensure that your applications can be called by other applications.

There are two possible approaches to this problem:

Rewrite your application to provide it with an API that other applications can call.
Create a communications adaptor that acts as an intermediary between the application and other applications.

Although rewriting an application to add APIs may seem like a more elegant solution than creating a series of bolted-on connectors or adapters, the time, effort, and risk associated with rewriting older applications makes the connector approach a preferred solution in most circumstances. However, for widely distributed and diverse applications, the distribution and management of connectors can generate significant management overhead.

You also need to determine whether the communication between applications will occur directly on a point-to-point basis, or through an intermediary hub. For more information, see "Making Application Integration Scalable" in Chapter 1, "Introduction."

Communications-Level Capabilities

Table 2.3 shows a number of capabilities that are required for effective communications integration.

Table 2.3: Communications Capabilities
Capability	Description
Message Routing	Controls the routing of messages from the source to the target application.
Message Delivery	Determines how messages pass from the source application to the target application.
Message Queuing	Implements reliable messaging.
Message Forwarding	Provides serial message handling.
Message Correlation	Ensures that reply messages are matched to request messages.
Message Instrumentation	Ensures that the messages are processed in the order that was originally intended.
Addressing	Determines where messages should be sent.
Transactional Delivery	Groups together messages, sending and receiving them within a transaction.
File Transfer	Moves files between applications.
Serialization/Deserialization	Converts data to and from a flat structure to send it over the network.
Request/Reply	Waits on a request sent to the target until a response is received.
Batching/Unbatching	Collects messages together for transmission and separates them at the destination.
Encode/Decode	Ensures that applications using different encoding methods or code pages can communicate.
Connection Management	Establishes a logical connection between applications in a request/reply scenario.

Not all of these capabilities are required for every application integration scenario. This guide discusses how to determine which capabilities are required in your environment.

Note

For a more detailed description of each capability, see the Appendix, "Application Integration Capabilities."