Current Technologies

Most of the integration goals that I have discussed can be addressed by current technologies. However, there are some serious limitations with these technologies that continue to make life difficult for developers. Just to clarify this point, developers may be up to taking on challenging projects, but in the past these projects typically have had a high risk factor and therefore had a tendency to fail. To better understand the risks, let's take a look at some of the most popular methods of data exchange in the IT world today.

Data Format

In the past, companies have struggled to exchange data because of the differences in data format preferences. Few tools were available to automate the translation of different formats, and those that did exist were weak solutions that needed a lot of customization. Microsoft's BizTalk Server has emerged in this space within the last two years . (For more about BizTalk Server, see Chapter 11.)

Two well-known solutions for data formatting have prevailed, although each has its own problems and limitations. The ASCII file format, which remains popular today, includes ASCII delimited files and ASCII fixed-width files. There is no standard way to format these files or even the values within the files. The implementation of the file format depends entirely on the business partners who are trying to exchange data. These businesses need to work closely to develop custom applications to load and handle each other's file format.

ASCII delimited files are basically text files, using the ASCII character-set , that have some form of delimiter between each field. The delimiters could be any ASCII character, though the popular delimiters are a comma or a tab. In some cases, text qualifiers like double quotation marks (") are used to specify the beginning and end of textual information. Using text qualifiers prevents textual information from being broken into separate values when a delimiter appears in the text.

In an ASCII fixed-width file, each column of data has a fixed width. A file of this type needs to be accompanied by a format file (also known as a header file ) that describes the width of each column. Again, there is no standard for formatting the fixed-width or format files.

A second solution is the use of specific file formats. Companies can exchange data using file types such as Microsoft Excel, Microsoft Access, IBM DB2 databases, or other proprietary options. A large amount coding is required to create "bridges" between the consuming applications and the application format of choice.

(At the dawn of the Internet age, in 1995, Microsoft was seriously planning to make the Word document a standard Internet format. Luckily for Microsoft, this idea was quickly dropped.)

Communication/Data Transmission

Sending and receiving data between business partners has been complicated by the fact that the available methods are slow, error prone, and typically need human intervention when an error occurs. Many data transmission types exist. Following are a few examples of how companies have exchanged data as well as the problems that they have encountered .

File Transfer Protocol (FTP) is a popular way to transfer files, although it is not highly automated and requires constant human intervention. This solution in not dynamic because it depends on files being sent and received. FTP facilitates the sending and receiving of files, but there is no specific approach to the actual data exchange. FTP also lacks sufficient security, which rules out any transferring of sensitive data.

Electronic Data Interface (EDI) has been around for many years now and has become a somewhat proven solution for data exchange between companies. EDI has its own issues, which include complexity and inflexibility. EDI is relatively expensive to implement, maintain, and deploy. This high cost tends to restrict the use of EDI to big organizations that support large transaction volumes and exclude small or new companies that want to exchange data with large organizations but do not have a budget for the software, hardware, and network connectivity required for EDI. If the implementation of EDI were somewhat easier and less expensive, EDI would allow trading partners, both small and large, to reap the benefits.

Recently, two competing technologies have emerged: Microsoft's Component Object Model (COM) and the Common Object Request Broker Architecture (CORBA). These technologies allow a more object-oriented approach to data exchange. COM and CORBA both support a remote procedure call (RPC) approach in contrast to the file-based transmission process. These technologies are also much less expensive to implement than other methods.

The Distributed Component Object Model (DCOM) enables communication between Microsoft COM-based applications residing on different machines. The DCOM protocol allows this communication to appear as if it were on the same machine. Internet Inter-ORB Protocol (IIOP)/Object Request Broker (ORB)/CORBA standards enable similar communication within Unix-based applications.

These technologies offer their own challenges and limitations. DCOM and CORBA are both platform-specific. To create a