Your organization probably uses many applications and services that were built over many months or years, as new business needs were identified. As a result, these applications probably are of different ages, were written by different people using different languages and technologies, reside on different hardware platforms, use different operating systems, and provide very different functionality. In fact, many of your applications often have very little in common at all, resulting in isolated functionality and multiple instances of the same data. For your organization, these conditions can result in redundant activities, higher costs, and inefficient response to your customers.
If you have read this far, your organization has probably identified a business requirement for applications to work together. Just as employees have to work together to meet business goals, your applications need to do the same.
This guide defines application integration as follows:
Application integration is the secure and orchestrated sharing of processes and/or data between applications within the enterprise.
Although this definition restricts integration to sharing within the boundaries of an enterprise, the guidelines described in this document also help in providing integration between different enterprises.
Effective application integration can provide your organization with the following important business benefits:
Allowing applications to be introduced into the organization more efficiently and at a lower cost
Allowing you to modify business processes as required by the organization
Providing more delivery channels for your organization
Allowing you to add automated steps into business processes that previously required manual intervention
Application integration can be broadly categorized into three types:
Manual application integration
Semi-automated application integration
Fully automated application integration
Most environments involve a combination of all three types. The following paragraphs describe each type in more detail.
Manual application integration requires people (employees and customers) to act as the interfaces between applications and enable the integration between them. This form of application integration is very common. As an example, think of a customer service department that takes information from the public. People may enter the same information into multiple systems and read information from those systems to respond to customer requests. In other cases, a person may need to read customer information from one database, and then reenter it into another database used for another purpose.
This form of integration requires very little technology investment. It becomes more complex, however, when your organization becomes more complex, and can lead to inaccuracies in data. As the amount and complexity of your data increases, or as the number of applications increases, you will require more and more people to maintain such an environment. An environment that relies heavily on manual integration is generally very inefficient, and does not grow as easily as environments that use more automated techniques.
Semi-automated application integration combines human steps with some automation. The person may be involved in an area where the corresponding automated solution is too difficult or expensive to implement, or where the business requires a person to make decisions. For example, your business may require a manager to approve all expense claims. In this case, all of the steps before and after managerial approval may be automated, but a person is required in the middle of the process. In other cases, human intervention may be needed to transform data that is required in another system.
Semi-automated application integration generally requires more technology investment, but once that investment is made, you can often reduce the number of people involved in integrating your applications. Reducing human involvement in this manner usually reduces costs and increases reliability.
Fully automated application integration removes people from the business process entirely, although they are required to maintain the solution. This type of integration consists of applications communicating through a series of interfaces and adapters. For example, two databases might share data, which is automatically transformed and committed to the second database from the first with no human interaction.
Although fully automated application integration removes the dependency on people, such systems can be more expensive to implement and may not be practical for many business problems. In many cases, you will still require people to make business decisions, and often it is more efficient to have a person control a technical process as well. For these reasons, you should decide where fully automated application integration is appropriate on a case-by-case basis.
Each type of application integration has its own set of costs and benefits, as outlined in Table 1.1.
Application integration type
Higher labor costs that scale badly. Subject to human error.
Little change required from existing low-technology environment.
Higher technology costs to implement. Subject to design-time and run-time errors.
Lower labor costs. Scales better. Less subject to human error at run time. Faster processing.
Highest technology costs to implement. Subject to design errors.
Lowest labor costs. Not subject to human error at run time. Lose human decision making on business processes, but faster processing.
Almost all environments involve a combination of all three types of application integration. Of course, in many cases people add to the effectiveness in your business processes. Situations where people often prove essential to effective business processes include:
Business process exception handling. Many different events can lead to a business process exception. They can result from human error at some point in a process, faulty logic (in the applications or in application integration), concurrency issues, or errors in data received from business partners. Because it is not possible to estimate all the possible errors and their corresponding fixes, it is not possible to totally automate responses to business process exceptions. You need the ability to manually intervene in the business process in some cases and make the appropriate correction without being restricted by what the system is expecting.
Manual override of business rules. Your applications will use business rules that are specific to the context of the enterprise at run time. These business rules facilitate automated execution of your business processes. However, you will need people to intervene in these business rules occasionally—for example, when shipping costs are waived for a large order from a regular customer. A fully automated solution with no manual override cannot allow for exceptions to be made based on human interaction with customers.
Approvals at key points within a workflow. Workflows in your organization can often be automated as long as the values of selected parameters fall within predefined limits. Transactions with values exceeding these limits usually require human approval to make sure that they are reasonable. For example, even if the processing and approval of expense reports within a specified amount is automated, you should ensure that validation by appropriate management personnel occurs if the amount exceeds a predefined limit.
System breakdown. A fully automated solution is completely dependent on the proper functioning of every component application that is part of the overall solution. Manual intervention enables you to temporarily replace a broken link in the chain of integrated applications with human transfer of information. For example, you may need to manually enter orders in applications that are ordinarily received by Electronic Data Interchange (EDI) if the EDI network you access is temporarily down due to a power outage.
If you closely examine processes that involve people, you can identify areas where semi-automated and fully automated business processes can be used. By doing so, you can reduce the number of employees involved in application integration and increase your potential for cost savings and scalability in your organization.
An important part of making application integration scalable is to increase the number of automated steps and reduce the number of human steps. This generally involves creating interfaces between applications along with predefined logic that replaces human involvement.
Increasing the level of automation generally increases the amount of information traveling between applications without increasing the number of employees required to support the environment. The scalability issues, however, do not stop at simple automation. You should also consider the number of applications themselves and how integration occurs between them. For automated application integration, you have two choices:
Integration hub model
The following paragraphs discuss each of these in turn.
The point-to-point model describes a decentralized structure in which each application communicates directly with the other applications. This type of integration is most appropriate for organizations that need to integrate a few applications with a small number of services.
Figure 1.1 shows the number of connections required for point-to-point environments with three and 10 applications, respectively, when you need to ensure that all applications can communicate with each other.
Figure 1.1: Point-to-point communication among applications
As the number of applications and services increases, the number of interfaces and connections that need to be maintained in a point-to-point environment rapidly increases.
The maximum number of connections required is defined as follows:
∑x (x= 1 to n−1)
where n is the number of applications you need to integrate. The number of interfaces may be up to twice the number of connections (depending on whether they are one-way or two-way interfaces).
This means that for the three application example (Scenario A in Figure 1.1), up to three connections and up to six interfaces are needed, whereas for the 10 application example (Scenario B), up to 45 connections and up to 90 interfaces are required.
In a point-to-point integration environment, interfaces between the applications are usually written as business needs arise. The problem with this approach is the lack of consistency. The integration approach used generally depends on which integration approach the developer is most familiar with, what he or she knows about the individual applications, and so on. As more and more applications are developed or updated, additional interfaces must be created.
The same problem pertains when new business goals are defined that require applications to communicate with one another differently. Every change makes the environment more difficult to understand, until eventually the structure is so complex that it is almost impossible to manage effectively. Such a complex structure may even hinder the company's ability to strategically shift its business goals, due to the high IT costs surrounding the change. Defining application integration on the fly in this way can massively increase your medium-term and long-term costs for a shortterm gain.
The integration hub model provides a more centralized structure, in which an integration hub is placed between the applications, and each application communicates with the hub rather than communicating directly with other applications.
Each application needs only an interface and a connection to the integration hub. To simplify matters, the integration hub can rely heavily on existing standards, which means that either the interfaces already exist or the methodologies for writing them are well-defined.
The main advantage of an integration hub environment is scalability. Figure 1.2 shows the use of an integration hub, again in environments with three and 10 applications, respectively.
Figure 1.2: Applications connected through an integration hub
In Figure 1.2, a single connection is required to the integration hub, with interfaces potentially required at the application and the integration hub. However, many environments require only one interface (or none if the application already uses the standards supported by the hub). This configuration leads to a maximum of three connections and six interfaces for the three application example, or 10 connections with 20 interfaces for the 10 application example. In most cases, the number of interfaces will be significantly lower.
The integration hub model is significantly more scalable for integration environments with many applications. A typical large-scale organization has thousands of islands of information, involving thousands of applications. You simply cannot create individual interfaces for every point of interaction. Instead, the solution is to create an application integration environment that allows all of your applications to communicate in a logical, predefined way. This hub infrastructure enables you to modify or update elements much more easily, and to do so when the business requires, rather than when the preexisting technology dictates. It should also allow the organization to more easily change direction and to use the products and services it has to match evolving requirements.
Because interfaces are within the integration hub (and are usually standards-based), you do not need to rewrite them whenever new applications are introduced. However, the hub can be technically challenging to implement and may be too costly for more simple application integration environments. In addition, you may have to sacrifice some data complexity to ensure that each application conforms to the standards of the integration hub.
As you examine the specifics of your environment, you will need to determine whether a point-to-point model, an integration hub model, or a combination of the two is most appropriate. Realistically, the decision you make will based on cost and value. In many cases, application integration involves a combination of both models, using point-to-point initially, and then moving to the use of one or more integration hubs as complexity increases and when there are clear business benefits to doing so.