Implementing Application Integration

You can implement application integration in a wide variety of ways. Common choices include:

• Web services

• Extract, Transform, and Load (ETL)

• Communications message protocols

• Screen scraping

• Program calls

• Direct data access

• File transfer

• Human involvement

The following paragraphs discuss each of these choices in turn.

Web Services

A Web service is an application component that:

• Exposes useful functionality to other Web services and applications through standard Web service protocols.

• Provides a way to describe its interfaces in detail, allowing developers to build client applications that talk to it. The description is provided in an XML document called a Web Services Description Language (WSDL) document.

• Describes its messages by using an XML schema.

Many vendors have endorsed and have started to provide Web service capabilities in their platforms. Using Web services for integration ensures that your integration is based on open standards that are language neutral and platform independent, and helps to ensure that the technology remains relevant over time.

Web services rely on defined specifications to allow systems from various vendors to interoperate. Web services specifications, commonly referred to as WS-* specs, are in various states of acceptance. The Web Services Interoperability Organization (http://www.ws-i.org) was formed to provide a framework and a set of tools to ensure that different vendors' implementations of a particular WS-* spec are compatible.

For more information about the current Web services standards being defined and the location of the specifications, see Web Services Specifications on MSDN (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnglobspec/html/wsspecsover.asp).

Extract, Transform, and Load

Extract, Transform, and Load (ETL) is a data integration technology that allows you to consolidate critical information from multiple enterprise data sources. Data is extracted and transformed to fit the data model of the target data store and is then loaded into that data store. ETL technologies are typically used to load high volumes of information into data warehouses and data marts from multiple databases.

Communications Message Protocols

Communications message protocols, such as TCP/IP, HTTP, and FTP, define a standardized method of transport for messages exchanged between applications. They typically specify a standard set of headers as well as a way to package the content to be transported within the body of the message. Some protocols define the message headers for both the requests that the applications initiate and the corresponding responses. Your applications may use these protocols directly, or you may choose to build custom integration methods around the protocols.

Screen Scraping

Sometimes the business logic of an application resides in the presentation layer. In this case, you can use software packages that simulate the human user's interactions to interface with the application. Screen scraping is a noninvasive mode of integration and does not introduce any additional channels of interface with the application. You can implement this technology in two ways:

• Use terminal emulation software to intercept screen images after they have been formatted into the terminal data stream.

• Intercept the data before it is formatted for display purposes. This method generally provides better performance and requires less interaction than using terminal emulation software.

Program Calls

You can use program calls to access the internal resources of an application. Applications supporting program calls are typically multitiered and modular, permitting external programs to take the place of the internal user interface layer of the application. They have a predefined set of inputs (parameters) and outputs (responses) depending on their business context. If you modify these inputs and outputs, you may also have to make corresponding modifications within the applications that are already invoking them.

Direct Data Access

Direct data access involves making native database calls to application databases or writing data directly to the target application file systems. This approach requires knowledge of the data access language of the target database as well as knowledge of the data model used in the target application.

Direct updates to databases can make them vulnerable to data corruption and referential integrity violations. For this reason, many applications do not support direct data access. Instead, ordinarily you use their application programming interfaces (APIs) so that the calling programs can benefit from the business logic implemented within the applications.

File Transfer

File transfer involves integration between multiple platforms through transfers of files in batch mode. Typically, you move data between systems asynchronously using batch file transfers, and the flow of data is unidirectional. Changes in one system rarely affect other systems unless the structure, format, or semantics of the data being moved changes. File transfer is a cost-effective solution for transferring batch files between a wide variety of platforms. A number of features add to the reliability and efficiency of file transfer, such as scheduling, automatic restart, guaranteed delivery, encryption/decryption, and compression/decompression.

Human Involvement

Almost every integration environment still uses people to some extent. People may be involved in one or more of the following activities:

• Presentation. Presentation is the extraction and subsequent presentation of the information to be transferred through various media to a person. Presentation is accomplished through fax and e-mail messages, printouts, or visual display of the information on an application screen.

• Validation. People are sometimes required to visually inspect data to ensure that basic business rules are being followed. The validation process can be augmented by providing the people with a predefined set of valid values (for example, reference codes).

• Transformation. People are sometimes required to reformat or make semantic changes to the data without making changes to the basic content (for example, changing date formats or regrouping information across address fields to fix data entry errors).

• Data entry. Data entry is the act of reentering the data into the target application. Basic cut-and-paste techniques can be employed to minimize the time spent and errors incurred in the data entry process.