Integration with Custom Code

Now that we have XML documents as input, we need to create interfaces to our applications to automate the processing of transactions. We have three options here. The first is to write custom code and program the interface ourselves from scratch. Second is to create an infrastructure for supporting transactions and multiple interfaces using BizTalk Server. Third is to choose another off-the-shelf EAI product, which is an option I do not explore in this book.

If you choose a custom code approach, you can either focus on building a single point-to-point interface or on creating a more general-purpose infrastructure for integration. In essence, you would be building a lightweight version of what BizTalk offers, although probably without as sophisticated a user interface and administrative utilities.

Let's take a look at custom .NET interfaces. The interface must handle orchestration of transactions by preventing the same transaction from being entered multiple times and guaranteeing that no transactions are dropped.

You might guess that building an interface with custom programming would be more difficult and time consuming than using BizTalk. You would certainly be right if you were talking about creating lots of interfaces. For instance, a reseller dealing with many suppliers would easily recoup an investment in the BizTalk infrastructure. For a small number of interfaces, however, the case is more difficult to judge. Setting up BizTalk is not a small task or expense, and much of the analytical heavy lifting in terms of data modeling and business process automation is the same, regardless of the software tools involved. For all these reasons, you may find yourself building a custom interface. While you could use other languages and tools (such as C++ and COM+), we focus on .NET and its toolbox of languages to build the solution.

Figure 11.8 shows a typical legacy integration solution. This example is of a system that accepts data from a mainframe using flat files that are periodically generated by the mainframe. The task of creating the files is simple for a mainframe programmer, and they are placed in an FTP directory to be picked up. This task is scheduled (for example, data is written to the files every five minutes throughout the business day).

Figure 11.8. Legacy Integration Scenario

Microsoft Host Integration Server is another approach for integrating mainframe data. It provides the middleware to make a hierarchical or flat file mainframe data source appear as a relational data source to a consuming application.

The next step is to obtain the new files when they come in. While this step could be accomplished through polling, with a continuously running application periodically checking the FTP directory, a better approach would be to use the file change notification API, which triggers an event when a new file is written to a folder.

After the file change is detected , the file name is passed to the business logic component that will perform the import, and the file is copied to a directory on the receiving server. Next, the hard work of the import is performed: the file is read, the fields from the source are mapped to the destination, and the records are parsed. Because these flat files are fixed length, the field definitions are stored in the application and can be updated easily if the source file format changes. When files are successfully imported, they are transferred to the archive directory.

Imported files are entered into the online transaction processing (OLTP) database through application of the business logic rules. Here is another area in which the system differs from an interactive application. When data entry errors arise in an interactive system, the user is presented with an error message and given the opportunity to correct the error and attempt the data entry again. The system simply does not accept user data that is known to be invalid. Here there are no human users with whom to interact, and therefore errors must be handled differently. Erroneous records are written to an errors database for further processing, such as returning a batch of records to the originator.

The data access layer contains objects that enwrap calls to the relational database. It provides a higher level of abstraction for these data calls. This layer is responsible for adding, deleting, and updating records.

Building an interface to a legacy system is fundamentally just another instance of the n - tier application architecture. That is, the architecture consists of several components that can logically be divided into the data, business logic, and presentation layers. The bottom two layers , business logic and database, are also found in an interactive form-based application. Indeed, they are not aware of whether the data is coming from a web-form, an XML document, or a flat file generated by a mainframe. The difference lies in the user interface layer. For application integration, the user forms are replaced by objects that create records from files rather than from one page, field, or keystroke at a time.