The BizTalk Framework Initiative | XML Programming Bible

Most mature organizations have both legacy and new applications that were implemented independently and cannot easily interoperate directly because of differences in data formats, message structures, and protocols. B2B projects, such as supply chains, face equally daunting problems. Even if both suppliers and buyers shared the same enterprise systems, their data dictionaries, and other customizations probably evolved differently over time. The core problems in both scenarios are the lack of interoperable data definitions and encoding, protocols, and the knowledge of how to process the data across disparate systems.

Microsoft believes it has a solution that spans both a standard framework and a server suite of applications that help businesses describe their data and processes, as well as manage the messaging between integrated servers. This solution to some of the problems we discussed concerning EAI and B2B comes in the form of a set of schemas, guidelines, and software tools collectively called the Microsoft BizTalk Framework Initiative (BFI). Although the BFI was proposed and mostly designed by Microsoft, it is open to everyone.

The BFI consists of three main components:

BizTalk Framework A set of guidelines and an XML tagging scheme for the construction of readable and reliable messages.
BizTalk.org A community Web site that provides a standard industry schema repository and an information resource center concerning the BFI.
BizTalk Server A BizTalk Framework compliant software system and suite of applications for providing robust and reliable application integration using XML-based messaging.

The BizTalk Framework

One of the most difficult problems faced by businesses engaged in application integration is the lack of established message schemas for integration purposes. Traditionally businesses used the EDI for integrating applications. However, these systems are generally large and complex and usually expensive to set up and maintain. In addition, businesses might be using different operating systems, communication protocols, and programming languages. All of these factors make application integration between businesses difficult, and until recently, only big organizations have attempted integrating their business processes with trading partners. The BizTalk Framework hopes to solve these issues by providing an open framework that can be used to develop standard message schemas. The core focus of the BizTalk Framework is the definition of the BizTalk document structure.

BizTalk Documents

A BizTalk document is an extension of a SOAP version 1.1 message with BizTalk-specific SOAP header elements (called BizTags) for enhanced message-handling semantics. Instead of an RPC method body, however, a BizTalk document's <Body> element contains an application-specific and well-formed XML document containing business data. This embedded XML document is called a business document in the BizTalk Framework specification. Examples include a purchase order, invoice, product catalog, or any other business document.

Listing 7-13 shows sample_biztalk_document.xml, a sample BizTalk document. This document contains a purchase order placement request. This code is essentially a SOAP 1.1 compliant message with a header that contains BizTags and a body that contains an application-specific document.

Listing 7-13 sample_biztalk_document.xml: A sample BizTalk document.

 <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance"> <soap:Header> <eps:endpoints soap:mustUnderstand="1" xmlns:eps="http://schemas.biztalk.org/btf-2-0/endpoints" xmlns:biz="http://schemas.biztalk.org/btf-2-0/address/types"> <eps:to> <eps:address xsi:type="biz:OrganizationName">ACME </eps:address> </eps:to> <eps:from>  <eps:address xsi:type="biz:OrganizationName">CoreXML </eps:address> </eps:from> </eps:endpoints> <prop:properties soap:mustUnderstand="1" xmlns:prop="http://schemas.biztalk.org/btf-2-0/properties"> <prop:identity>uuid:74b9f5d0-33fb-4a81-b02b-5b760641c1d6 </prop:identity> <prop:sentAt>2001-08-21T04:54:00-10:00</prop:sentAt> <prop:expiresAt>2001-08-22T04:54:00-10:00</prop:expiresAt> <prop:topic>purchase_order</prop:topic> </prop:properties> </soap:Header> <soap:Body> <po:PurchaseOrder xmlns:po="http://mspress.microsoft.com/ corexml/"> <po:ProductID>BX0071801</po:ProductID> </po:PurchaseOrder> </soap:Body> </soap:Envelope>

The current version of the BizTalk Framework uses the http://www.w3.org/1999/XMLSchema-instance XML Schema URI instead of the current http://www.w3c.org/2001/XMLSchema-instance because the specification was released before XML Schema became a W3C Recommendation.

BizTags

The BizTalk Framework defines four main groups of BizTags. The first group represents the source and destination of the interchange. This is specified by the <endpoints> BizTag. This BizTag is mandatory and must appear exactly once, and it consists of <to> and <from> subelements, each of which, in turn, contains a single <address> subelement.

Next the <properties> BizTag contains document identify information and other properties that describe the document. This BizTag is also mandatory and must appear exactly once. The <properties> element is also used to control the scheduling of a document through its <sentAt> and <expiresAt> subelements.

Following the <properties> element, we have the <services> element. The primary purpose of the <services> BizTag is to allow applications invoked in message interchange to request receipt acknowledgment from the consumer. The BizTalk Framework supports two kinds of receipts: the delivery receipt and the commitment receipt. Both are BizTalk documents with the addition of a special <deliveryReceipt> BizTag for conveying information related to the receipts, such as the identity of the target recipient and the expiration date.

The <manifest> BizTag serves to catalog all the contents of a multipart BizTalk message consisting of either multiple business documents or attachments that might not be XML files, such as JPEG images. A multipart BizTalk message must be encoded in a MIME structure the follows the rules of the multipart/related MIME content type.

Finally we have the <process> BizTag that provides process-management information that can help the recipient service relate this exchange with an ongoing business process. Together, all these BizTags allow BizTalk documents to be used for reliable and efficient routing message interchange.

BizTalk.org

BizTalk.org is a Web site (http://www.biztalk.org) hosted and managed by Microsoft to promote the adoption of BFI. The site has two main purposes. First it serves as an XML schema repository. Second it's a community and resource center for those building and implementing the BizTalk Framework into their services or applications.

The BizTalk.org repository aims to facilitate the initial integration efforts of potential partners by providing a fully indexed database of industry-standard and tested XML schemas. Prospective trading partners can use the repository to search for schemas by keywords or industry-standard category identifiers. Upon finding the matching schemas, BizTalk.org can help a trading partner determine whether a particular schema is relevant to the partner's need by providing additional documentation that is included with each schema in the repository.

BizTalk.org is also an information and resource center for anything related to BizTalk in particular or application integration in general. In addition, BizTalk.org provides community news and forum services. For those of you working with the BizTalk Framework, this site should definitely be a bookmark in your browser.

BizTalk Server

Microsoft's BizTalk Server is an XML-based messaging server that implements the document exchange functionality requirements laid out in the BizTalk Framework specification. Several companies are currently working on software that is BizTalk Framework compliant, but at the time of this writing the only released product is BizTalk Server. You should note that BizTalk Server goes well beyond the call of duty in this implementation, which you will soon see.

BizTalk Server is designed for both users and developers. Being a member of the Microsoft .NET Enterprise Server family of products, it provides a COM-based API for interacting with other BizTalk Servers programmatically. It also provides a rich set of GUI tools to allow users to easily manage the integration of applications. For example, BizTalk Messaging acts as a standard gateway where all low-level messaging is performed for the sending, receiving, and transformation of documents across disparate systems.

BizTalk Server provides two main categories of services: Orchestration Services and Messaging Services. We will also cover document tracking in this section.

Orchestration Services

Application integration is more than just the ability to exchange and process messages exchanged between the participants in the business process, or workflow. What must be added to have full integration is the ability to determine the logical order of actions and the corresponding flow of the messages. Traditionally businesses used modeling tools and workflow diagrams to help document the types of relationships and dependencies involved. BizTalk Orchestration Services provide a runtime engine and a design tool to allow users to design and schedule the execution of these workflows.

BizTalk Server provides a Microsoft Visio 2000-based design tool, called BizTalk Orchestration Designer that can be used to describe the business process in terms of a sequence of message exchanges. The user draws a flowchart to describe the relationship between the specific actions performed when messages are exchanged. It is also used to define the flow of data between messages. This flowchart, called a schedule, effectively documents how data flows through the business process from message to message. Orchestration Services store a schedule in an XML document that uses a Microsoft-designed schema called XLANG (pronounced "slang").

We will not get into the details of configuring and using Orchestration Services here. You will find that information in Chapter 14.

Messaging Services

BizTalk Messaging Services include receiving incoming documents, parsing and interpreting the embedded processing rules, possibly performing any necessary format transformation, and delivering the documents to their destinations. BizTalk Messaging Services also provide a mechanism for ensuring the integrity, reliability, and security of the document-interchange process. BizTalk Messaging Services are composed of a Messaging Engine and a GUI tool, called BizTalk Messaging Manager, for configuring and administering Messaging Services.

The Messaging Engine places both incoming and outgoing messages in queues. As we have learned, message queuing can greatly improve the reliability and scalability of the whole document interchange process. BizTalk Server is designed to use an external MOM system to handle the actual tasks required for supporting robust message queuing. The default MOM used by BizTalk Server is, unsurprisingly, the MSMQ.

Sending Documents to BizTalk Server

BizTalk Server provides two different methods for submitting documents to the Messaging Engine. First you can conFigure BizTalk Server to automatically pick up inbound documents found either in a user-specified MSMQ queue or a filesystem directory. Alternatively, you can programmatically submit documents to the BizTalk Server through the IInterchange COM interface. Let's take a closer look at these two methods.

Receive Functions

BizTalk Server provides a built-in facility, called receive functions, that you can use to get documents into the Messaging Engine without programming. Receive functions are components in BizTalk that can be configured to poll a specific resource for incoming documents. When a document arrives BizTalk will automatically take it off the resource and submit it to the Messaging Engine. Right out of the box, BizTalk supports two types of receive functions: file receive function and MSMQ receive function.

The file receive function monitors a user-specified file-system folder for incoming files that match a user-specified filename specification. When a matching file is found the file receive function will automatically submit it to the Messaging Engine. When the file is successfully submitted it will be removed from the folder.

The MSMQ receive function polls a user-specified MSMQ queue for incoming documents. When a document arrives it will be retrieved from the queue and submitted to the Messaging Engine. If the document is successfully submitted, the MSMQ transaction is committed and the document will be removed from the monitored queue. Otherwise, the transaction is rolled back and the document is left untouched in the queue.

The IInterchange COM Interface

In some cases the receive functions might not satisfy your requirements for submitting documents into BizTalk Server. For instance, you might have an ASP.NET application from which you want to submit documents to BizTalk directly. In cases like this, BizTalk offers a COM-based API you can use to submit your documents to BizTalk programmatically. This API is accessible through the IInterchange interface.

The IInterchange interface is simple and contains only a handful of methods. Table 7-5 summarizes the methods exposed by this interface. Note that the IInterchange interface also supports a few methods for interrogating and managing documents found in the suspended queue.

Table 7-5 The IInterchange Interface

Method	Description
Submit	Submits a document to the Messaging Engine for asynchronous processing. The submitted document is placed in a shared queue to await processing by the next available BizTalk Server in the server group. This method accepts only a string variable as the document or interchange. No object, such as COM objects, and other data types are allowed.
SubmitSync	Submits a document to the Messaging Engine and waits for a response. This method accepts only a string variable as the document or interchange. No object, such as COM objects, and other data types are allowed.
CheckSuspendedQueue	Checks the shared suspended queue and returns a list of handles to documents or interchanges in the queue that match the specified criteria.
GetSuspendedQueueItemDetails	Obtains details of a document in the suspended queue.
DeleteFromSuspendedQueue	Deletes one or more documents from the suspended queue.

Listing 7-14, submit_biztalk.aspx, is an ASP.NET application that submits a document to BizTalk using the Submit( ) method. The application first prompts you for the name of the channel (we will explain what this is in the next section) and document contents. It then submits the specified document string to the specified channel in BizTalk.

Listing 7-14 submit_biztalk.aspx: An ASP.NET application that submits a document to BizTalk.

 <%@page language="vb"%> <% ' retrieve submitted channel name and document string dim sDocument as String = "" & Request.Form( "pDocument" ) dim sChannel as String = "" & Request.Form( "pChannel" ) %> <form method="post" action="submit_biztalk.asp"> <table borders="0" cellpadding="2" cellspacing="0"> <tr> <td valign="top">Channel</td> <td><input name="pChannel" value="<%=sChannel%>"></td> </tr> <tr> <td valign="top">Document</td> <td><textarea name="pDocument" cols="50" rows="15"> <%=sDocument%></textarea></td> </tr> <tr> <td></td> <td><input type="submit" value="Submit"></td> </tr> </table> </form> <% if ( sDocument <> "" ) then dim oInterchange as new Interchange dim sHandle as String ' submits a document to BizTalk for asynchronous processing sHandle = oInterchange.Submit( BIZTALK_OPENNESS_TYPE_NOTOPEN, sDocument,,,,,, sChannel ) end if %>

The BIZTALK_OPENNESS_TYPE_NOTOPEN enumeration value in the Submit( ) method call shown in Submit_BizTalk.aspx specifies that BizTalk should use a nonopen messaging port to receive the specified document. We will explain what a messaging port is in the next section.

We will not go over the details of the syntax of the Submit method and its parameters in this chapter. If you're interested you should read the BizTalk manuals or online help for further information.

Figure 7-11 shows a sample screenshot of the Submit_BizTalk.aspx application.

Receiving Documents from BizTalk Server

For applications to receive documents, BizTalk Server provides a set of transport services for delivering documents to their destinations. It offers four types of transport services:

File
MSMQ
Network protocols
COM-based Application Integration Components (AICs)

The network protocol transport services supported by BizTalk Server are standard Internet protocols including HTTP, HTTPS, and SMTP.

AICs are COM objects that implement the IBTSAppIntegration interface. The IBTSAppIntegration is an extremely simple COM interface that exposes a single method, ProcessMessage( ). AIC eliminates the need for BizTalk Server to natively understand the myriad of proprietary protocols invented by different application vendors. It also allows the application to be extensible in terms of interfacing with third-party applications. For example, instead of providing native support for SAP (a popular Enterprise Resources Manager System), an SAP AIC can be developed to allow BizTalk Server to send documents to SAP.

Figure 7-11 Running Submit_BizTalk.aspx.

The Messaging Manager

The BizTalk Messaging Manager, shown in Figure 7-12, is a GUI tool that allows the user to manage the exchange of documents by configuring the various components of Messaging Services.

Figure 7-12 The BizTalk Messaging Manager.

The Messaging Manager window is divided into two frames. The left frame provides navigational control to allow the user to select the object type and to search for objects of the selected type to manage. The right frame lists the results of the search depending on the object type. Double clicking on one of these items opens the wizard associated with the object type.

The Messaging Manager has wizards and properties dialogs for creating and configuring the objects that collectively control the operation of Messaging Services. These objects are organizations, document definitions, envelopes, messaging ports, and channels. Figure 7-13 shows the relationship between some of these objects.

Organizations

Organizations represent the trading partners with which you exchange documents. BizTalk has a special organization type, called home organization, that is used to represent your business.

Organizations have identifiers that help the Messaging Engine determine the configuration to use for processing documents. Note that BizTalk's only interest in organizations is in organizing configurations. Therefore, you can use organizations to represent departments, workgroups, or any entity that you want to have available for defining messaging configurations.

Figure 7-13 BizTalk Messaging Manager objects.

Figure 7-14 shows the Organization Properties dialog. The General tab is used to specify a name and descriptive comments for an organization. The Identifiers tab lists all the identifiers associated with an organization.

Figure 7-14 The Organization Properties dialog.

Document Definitions

A document definition is used to define a specific type of document that is processed by BizTalk. A document definition provides a pointer to a specification that, in turn, is used to define the document structure, type, and version. You will learn more about specifications in Chapter 14.

Figure 7-15 shows the Document Definition Properties dialog. It has three tabs: General, Global Tracking, and Selection Criteria.

Figure 7-15 The General tab of the Document Definitions properties dialog.

The General tab is used to define the common name and the reference to the associated document specification. For the document specification reference you must specify a WebDAV URL. You can use the Browse button to navigate and locate the appropriate WebDAV repository.

As we will see in a later section, BizTalk provides a messaging tracking capability. By default, BizTalk automatically stores certain basic information, but you can extend this to include other fields specific to your application. The Global Tracking tab, shown in Figure 7-16, lets you define the fields that you want to track.

Figure 7-16 The Global Tracking tab of the Document Definitions properties dialog.

The leftmost panel displays a tree-view representation of the fields provided in your document definition. To select a field to track you click on the appropriate button in the middle depending on the data type of the field. Once a field is selected, it will be included in the rightmost panel. This panel lists all the fields to be tracked.

The Selection Criteria tab is used for EDI documents only. It contains a list of name-value pairs that are used by BizTalk to uniquely identify the appropriate document definition for an inbound document. Please refer to the BizTalk Server manuals for further details about EDI documents and the Selection Criteria tab.

Envelopes

Envelopes are used to provide the Messaging Engine with information it needs for processing a document it carries. BizTalk supports the following six types of envelopes:

XML
ANSI X12 (an EDI format)
UN/EDIFACT (an EDI format)
Flat file (delimited and positional)
Reliable (BizTalk Framework)
Custom

Figure 7-17 shows the Envelope Properties dialog. The Envelope name is used to identify this Envelope specification when used in other configuration tasks. The Envelope format drop-down box lists the formats supported by BizTalk. When you select Custom XML, FLATFILE, or CUSTOM, you're required to specify a document specification that describes the structure of the document for which this envelope is intended.

Figure 7-17 The Envelope Properties dialog.

Messaging Ports

A messaging port can be thought of as a set of rules that trading partner organizations accept for exchanging documents. It includes destination, transport type, security, and envelope information.

You create a new messaging port by selecting the New Port menu item from the File menu. The BizTalk Messaging Manager will then display the Messaging Port wizard. The first detail you need to specify is a name and descriptive comments for this new port. When you click the Next button you will be shown a Destination Organization page similar to that shown in Figure 7-18.

You need to specify the destination organization and transport type first. For the destination you can either specify an open destination or an explicit organization by selecting from the Organization drop-down list box. An open destination is one where the destination information is determined at runtime based on the information contained in the message itself.

After that, you need to specify the primary transport the Messaging Engine will use to send the processed documents to the specified destination organization. You can use the Browse button to select from a list of available transport protocols such as HTTP, SMTP, and AIC. This option is not available for open destination.

Figure 7-18 Destination Organization page of the Messaging Port wizard.

The Service window option is used to specify an optional time window within which the transmission is restricted. Times are specified to the nearest hour using the From and To edit boxes.

Once you've completed specifying the information for the Destination Organization, clicking the Next button takes you to the Envelope Information page. This is shown in Figure 7-19.

Figure 7-19 The Envelope Information page of the Messaging Port wizard.

Here you select the envelope to associate with this messaging port. The Envelope drop-down list box lists all the envelopes known to the Messaging Manager. You can also click the New button to create a new envelope at this point.

The next and final page is the Security Information page, which is used to specify security-related information for this messaging port. This is shown in Figure 7-20.

The Encoding drop-down list box specifies the encoding type used by messages associated with this messaging port. For text messages, this is usually none. For other types, you can select either the MIME or Custom type. The latter type requires a custom encoding component. Refer to the BizTalk manuals for information about creating and using custom encoding.

The Encryption drop-down list box lets you instruct BizTalk to protect all the messages going through this messaging port by encrypting their contents using the specified encryption method. Currently BizTalk only supports the Secure Multipurpose Internet Mail Extensions (S/MIME) encryption type. You will also need to specify the digital certificate for your partner.

Channels

Channels are the primary object in Messaging Services. All the other objects we have learned about so far—organizations, document definitions, envelopes, and messaging ports—are used to support the operation of channels. A channel forms a single self-contained and named object that BizTalk uses to conFigure Messaging Services to process a document it receives.

Figure 7-20 Security Information page of the Messaging Port wizard.

Configuring channels is beyond the scope of this chapter. Please refer to the BizTalk Server manuals for further information about creating and managing channels.

Document Tracking

One detail that might be obvious is the need to track messages and documents as they flow through the system. One key advantage of BizTalk Server is that document tracking is a built-in service. Even without any programming on the user's part, BizTalk can be configured to automatically capture and display user-defined information that can be used to track the status of individual messages transiting the messaging system.

Capturing Information to Track

Metadata associated with an interchange, such as source and destination information, document type, and date and time parameters, is automatically stored and available for tracking by BizTalk Messaging Services. However, if you want to be able to track information extracted from messages submitted to BizTalk, you will need to conFigure the BizTalk Messaging Manager to capture this information.

These tracking options can be set at the following three levels:

Server Group
Channel
Document Definition

You've already seen how you can specify tracking options at the Document Definition level when we discussed the Document Definition object in a previous section.

For the server group tracking options you will need to start the BizTalk Administration console, select a server group, and open the associated properties dialog. This is shown in Figure 7-21.

Figure 7-21 Server group document tracking options.

Four tracking options can be configured globally for a specific server group. The Enable document tracking check box turns tracking for this server group on and off. The other three tracking options are only available when the Enable document tracking option is set.

When the Log incoming interchange and Log outgoing interchange tracking options are enabled, BizTalk will keep a copy of the input and output documents in their original native format. This is a great option for those cases when you need to fulfill legal or standard requirements or both to keep copies of all electronic business transactions.

The last option, Log the original MIME-encoded message, instructs BizTalk to store S/MIME encoded documents in the Tracking database in their original, preencoded, format.

Displaying Tracking Information

BizTalk provides a Web-based GUI tool to enable users to track documents visually. BizTalk Document Tracking provides many different search and sort options to enable the user to easily locate and format the query result for specific groups of documents that the user wishes to track.