4.1 A Brief History of EDI

---

EDI has been in use since the mid-1960s, when a group of railroad companies formed an organization to improve the quality of transportation data being transferred from company to company. Retail companies also saw the opportunity to use EDI as a way of integrating with their suppliers and started an initiative to build industry-specific EDI solutions. Over the years other standards started to emerge, and in 1985 EDIFACT (EDI for Administration, Commerce, and Transport) was created through the United Nations.

4.1.1 EDI and BizTalk Server

Although there are some similarities between EDI and BizTalk Server, there are also some notable areas in which the approaches differ.

EDI has gained widespread acceptance and, since the standard has been established for a number of years, many large organizations use EDI as an intrinsic part of their everyday business. There are limitations with EDI, notably the cost of setup, and maintenance can be high as can the use of the value added networks (VANS) necessary to 'hook' into EDI systems. In addition, EDI is a useful tool for server-based connections outside of a business, but would not be used as an organization's enterprise application integration platform (see Figure 4.2).

Figure 4.2: Hub-and-spoke EAI.

BizTalk Server supports EDI formats and receipting, which is sufficient for interoperating with many EDI-based systems. BizTalk Server does not currently provide a fully fledged EDI-based environment and therefore does have a number of limitations for those wishing to undertake a full EDI-based implementation with the product. A number of these limitations can be overcome using customized code within BizTalk Server, but that is additional functionality that an organization would need to purchase separately.

As I guess you would expect by now the interchange of data uses XML with the messages containing self-describing data. This message can be unravelled by BizTalk Server, and, if necessary, appropriate business processes can be executed on receipt of the message. In addition, BizTalk Server comes with a set of tools that enables the XML message to be converted into a number of other document formats, including EDI850, UN/ EDIFACT, and comma separated values (CSV), providing the important integration hooks (see Figure 4.3).

Figure 4.3: BizTalk Server supports a number of different file formats and submission routes.

4.1.2 BizTalk Server and Web Services

It is important to understand the differences between Web services and Biz- Talk Server functionality because they can be confusing. In essence, as we can see in Figure 4.4, BizTalk is a useful mechanism for transferring business documents and integrating with existing ERP-type systems. Web services, on the other hand, are a way of exposing application functionality across the Internet using common protocols such as SOAP and HTTP/ XML. Web services are explained further in Chapter 9.

Figure 4.4: Comparing Web services and BizTalk Server.

4.1.3 BizTalk Server Messaging Services

One of the useful features of BizTalk Server is that the application that integrates with BizTalk Server never really knows that BizTalk Server is actually there. All the source application or system needs to do is produce a file containing the data, which is then picked up by BizTalk Server. This could be as simple as placing a file into an appropriate network directory. This is heart warming to many EAI practitioners because the interface is noninvasive-that is, you don't have to mess around with the internals of the source application since most systems are capable of providing simple data extracts on a regular basis. Failing that, there is a screen scraping option but that is functionality outside of the scope of BizTalk Server and something you will need to implement yourself using the appropriate software tools.

The file submission need not be in real time. BizTalk Server is just as happy to work asynchronously and pick up files based on a specific time rather than as soon as they appear. Again, this will be dictated by the business model you are working with. This also gives some protection from network failure since after a network outage, BizTalk Server will revisit appropriate directories and pick up any outstanding files.

To configure a document pickup, the BizTalk Server administrators use the Receive functions in the BizTalk Server administration tool. The dialog allows you to set the server and polling location details (see Figure 4.5).

Figure 4.5: Setting up Receive function properties for an advanced shipping notice.

The Advanced tab allows the document destination to be set up. As well as picking up files from a directory, BizTalk Server allows files to be submitted via HTTP, message queuing, SMTP, and COM from within an application.

The file destination is called a message port in BizTalk Server, and here you can set up where the file goes; if the file is to go to multiple destinations; you can setup distribution lists (see Figure 4.6).

Figure 4.6: Messaging port properties using an SAP integration component.

BizTalk Server also supports the use of channels. A channel determines what happens to a file before it reaches its final destination. Typically a file will need to be manipulated, transformed, or signed at some point on its journey, and this is what the channel is designed to do (see Figure 4.7).

Figure 4.7: Setting new channel properties.

4.1.4 Document Definitions and the BizTalk Editor

Given the choice of using Notepad or a decent tool to create XML documents, most people would choose a decent tool. Luckily, BizTalk Server comes with such a beast called the BizTalk Editor. This tool is designed to reduce the chore of writing XML script and, hopefully, the number of errors that are bound to creep into any heavy manual authoring process (see Figure 4.8).

Figure 4.8: The BizTalk Editor.

The BizTalk Editor can be used to display and edit any BizTalk Server- supported document format, including EDI or CSV files. Once a specification has been set up, it is saved to the WebDAV repository. WebDAV is the World Wide Web Document Authoring and Versioning standard designed to support collaborative authoring across the World Wide Web. It is an open standard maintained by the Internet Engineering Task Force, or IETF. The document definition is the specification you built in the BizTalk Editor combined with a name given to it within BizTalk Server. With this approach you can have multiple document definitions, all based on the same specification.

4.1.5 Business Process Orchestration

Once the document definitions and channels have been configured, you need to be able to define what happens to the files when they are received by BizTalk Server. In BizTalk Server, this is known as business process orchestration. Based on Microsoft Visio, the BizTalk Server orchestration tool allows a nontechnical business expert to define what happens to the file graphically by drawing the images on the screen. All of the traditional business flow chart tools are available, so that actions, decisions, whiles, and transactions can be modeled graphically and in a fairly easy-to-use way. Multiple activities can be tied up into single transactions if failure of any one component must lead to a rollback of a piece of work, similar to a database transaction (see Figure 4.9). The business expert can then hand the process workflow over to a developer, who is then responsible for translating the workflow into a technical equivalent.

Figure 4.9: Business process orchestration.

A lot of the coding work has been removed since the developer can use a set of predefined components and drag-and-drop them onto the developer side of the screen. The three fundamental shapes are the COM component, message queuing, and BizTalk messaging shapes, and these will probably form the basis for most systems a developer will design.

4.1.6 COM Component Shape

This allows BizTalk Server to work with other preexisting COM components or other applications. When this shape is used, the COM component binding wizard is invoked and a set of questions is run through. If you are using COM+ components, you will be presented with additional questions associated with the transaction support required for your component.

4.1.7 Message Queuing Shape

This allows an XLANG schedule to communicate with another XLANG schedule or application by placing messages into a queue, which is then read. Using the message queue shape will invoke the message queuing binding wizard, which will take the developer through the steps of configuring how the queue will be used. If the nature of the queue is likely to change, then a dynamic queue can be configured, or alternatively, if the queue is unlikely to change, then a static queue is set up. Messages can also be filtered to ensure that they contain sender information, if required, prior to being submitted to the queue.

4.1.8 BizTalk Messaging Shape

This is the BizTalk messaging services shape used to configure message exchange between BizTalk orchestration services and the messaging engine. When used, the BizTalk messaging binding wizard appears, taking you through the configuration settings required. This includes the communication direction, which tells BizTalk Server if it is sending or receiving documents, and channel information about where to place the documents.

Other tasks can be custom built as script components using any of the standard scripting tools, such as VBScript, but the chances are that you will (and maybe should!) use the COM component for most of the external connections you make.

To join the two sides of the diagram together a line is drawn in a typical Visio way that hooks the analyst's view into the implementation view. The BizTalk Server communication wizard will then prompt for some extra details on how the shapes will talk to each other. Finally, the data page is completed; this determines how the business document will flow from one process to another.

The business process orchestration drawing is formally known as the XLANG drawing, and the drawing plus developer implementation is the XLANG schedule. The schedule is written to a file in the XLANG language, which uses XML to represent the business process and technological implementation. To convert the diagram into the final XLANG language representation there is an option under the file menu called 'make XLANG filename.skx,' which does this for you. BizTalk Server components are shown in Figure 4.10.

Figure 4.10: BizTalk server components.

4.1.9 The BizTalk Mapper

BizTalk Mapper is the tool used to transform an XML document from one schema to another. It uses standard XSL Transformations (XSLT), which is a W3C-approved language for transforming XML schemata. The tool provides a neat interface on top of this language, so all a developer needs to do is drag-and-drop links from one schema to another.

If this simple one-to-one relationship is not enough, BizTalk Mapper comes with functoids. These delightfully named transformation elements allow all sorts of processing to happen as part of the transformation process, such as string manipulation, mathematical functions, and scientific operations. These are very similar to Microsoft Excel functions, and the BizTalk Mapper ships with 60 or so of these in the box. If you want a customized function, there is a script functoid, which allows you to write custom code in VBScript. Functoids can be linked or cascaded, so that results from one can be passed to another (see Figure 4.11).

Figure 4.11: Biztalk Mapper with three functoids to assist the transformations.

4.1.10 BizTalk Server and Protocols

As already discussed, BizTalk Server supports a number of protocols that allow it to receive and transmit documents between organizations.

Deciding which protocol to use will depend on a number of factors. For example, what protocols does your partner organization support? Are you using a private network or the Internet? How secure does the transmission need to be? And so on. Probably the most popular protocol will be HTTP or its secure equivalent, HTTPS, simply because these are available to all across the Internet.

Probably the most straightforward way of receiving HTTP inbound messages is to use an ASP script, which accepts the incoming document and then sets it to one side in a file directory, ready for collection by Biz- Talk Server.

Using this asynchronous approach, the document sender is simply sent an acknowledgment that the document has arrived safely, whether or not BizTalk Server has subsequently picked up the message for processing. Although this may appear fragile, it is more than suitable for many business scenarios since BizTalk Server will pick up and process the message in what I tend to call 'real-enough time' rather than real time.

Forcing BizTalk Server to operate in a real-time mode by submitting the documents directly via the COM interface is fine, but you will need to accept that should the system fail, the document originator will not receive an acknowledgment when he or she submits the document, and the system would be seen to have failed. Again, the business needs to decide which scenario best fits its objectives.

To improve on this directory/temporary storage-based document submission, BizTalk Server comes ready to use Microsoft Message Queue (MSMQ). Instead of the document being parked in a directory location, it will be posted to a message queue and then posted to BizTalk Server using a message queue Receive function.

4.1.11 Application Integration Components (AIC)

Since the release of BizTalk Server, there have been a large number of third parties that have built AICs to connect BizTalk Server with a large range of back-end business systems. The AIC allows the exchange of documents between BizTalk Server and the business system, managing transport, security, and serialization where appropriate.

The AICs are COM objects that BizTalk Server will call when delivering data to the back-end system. Assuming that the message port is configured to call the AIC, each time BizTalk Server needs to send a document, the component is automatically instantiated and the data passed back. The AIC is responsible for all of the underlying communication to the back-end system, using whatever API calls or database access it needs.

There are two models for building AICs:

1. Pipeline components are used when the component requires configuration properties. These components have been included to support integration between BizTalk Server and Commerce Server/

   Site Server. Pipeline components written for the Commerce Server commerce interchange pipeline are, therefore, compatible with BizTalk Server components written this way.

2. Lightweight components use a far simpler interface, which is much easier to build than pipeline components . By default BizTalk Server will query the component to see if the lightweight model is being used, and if it is not, it will then query for pipeline interfaces.

Once built the AICs need to be registered so that BizTalk Server understands where the components are and how they can be used. The AIC can be registered as either an in-process or out-of-process component. In- process delivers better performance, but if the component should fail, then Biz- Talk Server will probably terminate, so the usual out-of-process registration is normally used.

4.1.12 SAP and MQSeries Integration with BizTalk Server

Integrating BizTalk Server with existing line of business systems is an important consideration for many users of BizTalk Server and application integration components; many line of business solutions can be obtained from third parties.

4.1.13 MQSeries

The objective of the Microsoft MQSeries adapter is to allow BizTalk to send and receive messages to MQSeries, particularly when running on non- Windows platforms. The adapter has been designed to be easy to deploy and to support the transactional delivery of all messages. Any messages sent to the MQSeries adapter are guaranteed to be delivered, will only be delivered once, and can be converted from UNICODE to ANSI.

The Receive function for MQSeries runs as a BizTalk Server service and supports up to 100MB of messages at any one time. Distributed transactions are supported, and multiple MQSeries queue managers can be accessed at any one time. An MQHelper is needed if you wish to support distributed transactions, and this will run under dllhost.exe on the MQSeries server (see Figure 4.12).

Figure 4.12: MQSeries and BizTalk Server architecture.

4.1.14 SAP

The SAP adapter provided by Microsoft is designed to offer a simple solution to integrate BizTalk with SAP. It can be configured in a short time and does not need any software to be installed on the SAP system. The adapter is designed to allow BizTalk Server to send and receive SAP IDocs and will support any SAP IDoc format (see Figures 4.13 and 4.14).

Figure 4.13: Receiving an IDoc from SAP.

Figure 4.14: Sending an IDoc to SAP.

The adapter supports XML schema generation and allows the SAP Business Objects Repository (BOR) to be accessed via Remote Function Calls (RFCs). The IDoc structures are retrieved from the BOR and converted to document specifications. Multiple SAP systems can be sorted from one Biz- Talk Server, and the IDoc delivery is guaranteed since the adapter uses the Distributed Transaction Coordinator between com4abap, SQL Server, and MSMQ to deliver documents.

Setting up the SAP adapter is reasonably straightforward. Accounts need to be set up for com4abap and the adapter to access SAP. RFC ports are then configured, and document destinations, customer profiles, and SAP business processes are also configured.

---