What Is BizTalk Server 2004?

Microsoft s Jupiter vision enables a connected business by bringing together Microsoft s e-business servers into a single unified environment. The goal of connecting these server products is to provide improved experiences for developers, information workers, and IT professionals through common integration with Visual Studio.NET 2003, the Microsoft Office System, integrated security, deployment, management, monitoring, and Web Services for interoperability in the enterprise. The first version of Jupiter, BizTalk Server 2004, is built on these design themes and lays the foundation for the later release of the larger Jupiter vision.

BizTalk Server 2004 is a full-featured integration server that enables the automation and management of business processes that connect internal systems with people and trading partners . It provides an orchestration engine along with a set of shared services that are designed to make it easier to orchestrate dynamic business processes inside and between organizations. BizTalk Server 2004 focuses on the following areas of integration:

• Business process management

• Human-based workflow

• Business rules

• Single sign-on

• InfoPath integration

A well-managed and orchestrated set of business processes provide operational and competitive advantages to enterprises . Business processes are a core value of any enterprise. Built on top of the .NET Framework, BizTalk Server 2004 is composed of two separate services:

BizTalk Orchestration: Allows the execution of a business processes

BizTalk Messaging: Transforms and routes data between business processes

These two services are combined together as shown in Figure 7.1 to receive, route, process, and send messages. Orchestration services enable you to design, deploy, and manage business processes through an XML dialect called XLANG. One of the other advantages of BizTalk orchestration is that you can include concurrent and long-running state management processes.

Figure 7.1: The BizTalk Server 2004 architecture.

BizTalk messaging enables the receiving of incoming documents and parses them to extract specific formatting information. This may include things like key identifiers or specific routing instructions and rules used to deliver documents to their destinations. Messaging also allows you to map data formats to other formats and apply security to ensure data integrity.

The Architecture Overview

The combination of orchestration and messaging into a single subsystem enables a shared common data store for maintaining both process state management and messaging. During the actual execution of a process, the messaging components are designed to handle the transport and mapping, while the orchestration executes the actual process.

All communications with a deployed business process occur through adapters. Adapters are the implementation of a transport protocol; e.g., HTTP or Simple Mail Transfer Protocol (SMTP). BizTalk Server supports the following native adapters:

File Receive Adapter: The file receive adapter is used to read messages from files and submit them to the server. The receive adapter reads the file and creates a BizTalk Message object, so that the message can be processed by BizTalk Server. While the file is being read, it is locked to ensure that it is not modified while the message is being created.

HTTP Send Adapter: The HTTP send adapter retrieves messages from the server and sends them to a destination URL on an HTTP POST request. The HTTP send adapter gets the message content from the body part of the BizTalk Message object. All other parts of the BizTalk Message object are ignored by the HTTP send adapter.

SMTP Adapter: The SMTP adapter is used to exchange information between a server running BizTalk Server and other applications by means of the SMTP protocol. BizTalk Server can send messages to other applications by creating an e-mail message and delivering it to a specified e-mail address. Internally, the SMTP send adapter creates an SMTP-based e-mail message and sends it to a target e-mail address that is configured as a property of the SMTP adapter.

SOAP Adapter: The SOAP adapter is used by BizTalk Web Services when receiving and sending Web Service requests . It enables orchestrations to be published as Web Services and allows you to consume other external Web Services.

MSMQT Adapter: The MSMQT adapter is the native Microsoft Message Queuing (MSMQ) adapter in BizTalk Server. MSMQ is a stand-alone reliable messaging service, with its own administration tools and message store. MSMQT works the same as MSMQ from a network perspective; the main difference is that instead of being sent to a queue, messages are sent to a receive location.

FTP Adapter: The FTP adapter enables the exchange of files between BizTalk Server 2004 and FTP servers.

SQL Adapter: The SQL adapter is used to exchange data between BizTalk Server and an SQL Server database. This adapter can pull individual records from one or more data tables, save the records as one or more XML messages, and pass the message to a line-of-business application through BizTalk Server. You can also use the SQL adapter to move large amounts of data to or from the SQL Server database or the Data Warehouse as part of a complex BizTalk Server messaging or orchestration solution. In addition, you can use the SQL adapter to update and delete SQL Server tables by using parameterized SQL statements or stored procedures.

Adapters are added or configured through the BizTalk Administration Console, as shown in Figure 7.2. Enterprise Single Sign-On (SSO) enables users to sign on only once when interoperating with heterogeneous systems. This allows BizTalk adapters to provide the appropriate user ID and credentials to back-end systems based on the user s Windows 2000 credentials. Once authenticated by Windows, the user does not need to provide any additional credentials to connect to a back-end system.

Figure 7.2: Managing the native adapters through the BizTalk Administration Console.

By default, SSO is available for the HTTP and SOAP adapters. For the HTTP adapter, the send port and receive location can be configured to use SSO; for the SOAP adapter, only the receive location can be configured. By default, SSO is disabled and can be configured using the BizTalk Explorer Console. Authentication in SSO relies primarily on Windows and the Windows groups created in Active Directory. All operations completed by a user or administrator with SSO require that the user or administrator first be authenticated using Windows authentication.

Note  

The BizTalk Framework also provides an extensibility model that many third-party vendors can use to extend or write their own adapters for specific line-of-business applications.

Each of the specific receive location and send ports, which act as the addresses on an adapter, are configured using the BizTalk Explorer, as shown in Figure 7.3. BizTalk Explorer is available within the Visual Studio.NET 2003 IDE.

Figure 7.3: BizTalk Explorer, which is part of Visual Studio.NET 2003.

As a message is received through the receive adapter, it is then processed through a receive pipeline. This pipeline takes the incoming message, disassembles it from its native format into an XML document, and generates one or more messages that can be processed by the BizTalk Server. Table 7.1 shows the four stages of the receive pipeline and what occurs during each stage.

Table 7.1: Stages of the receive pipeline.

Stage	Description
Decode Stage	Decodes or decrypts the incoming messages
Disassemble Stage	Parses and reviews the message to determine if the format of the message is recognized
Validate Stage	Validates the messages based on the recognized format
ResolveParty Stage	Maps the sender certificate to specific routing formats

MessageBox Database

At the core of the messaging and orchestrations is the MessageBox database, which stores all messages that an application can interact with. It also serves as a load- balancing mechanism used to distribute work to additional BizTalk Servers. When a BizTalk Server receives a message, the message is processed in a pipeline and then placed in the MessageBox database.

The incoming message has a context; that is, a set of properties associated with the message. For example, these properties may include the type of message or sender. BizTalk supports three types of properties: simple written properties, promoted properties, and predicate properties. Simple written properties are the base properties and messages defined within the system and business process. The promoted and predicate message properties are used to determine what business process is subscribed to this message, and whether the business process has the permissions necessary to receive the message.

The MessageBox database uses a subscription to determine which messages are distributed to which servers. Subscriptions are a list of stored predicate and service information. The predicate information is the criteria that must be met by a message, and the service information is what to do with the message that meets the criteria.

Different business processes expect different types of messages at different times. Subscriptions are criteria that describe which of these message types the business process will receive. Each subscription is a query against the context properties of messages stored in the MessageBox database.

When a matching subscription is filled, a business process receives the message from the MessageBox database and processes the message on an available BizTalk Host instance. Each BizTalk Host is a logical container for items such as adapters, receive functions, and orchestrations. A BizTalk Host performs the following functions:

Receiving: These items do the initial processing of messages after they are picked up in a receive location. When a host contains a receiving item, such as a receive location or pipeline, it functions as a security boundary, and the message decoding and decrypting occur directly in the host pipeline.

Sending: These items do the final processing of messages before they are sent out of the send port. When a host contains a sending item, such as a send port or pipeline, the host provides a security boundary, and the signing and encryption of a message occurs in the host pipeline.

Processing: These items process messages based on the instructions defined in an orchestration.

Note  

One BizTalk Host can contain multiple items that send, receive, and process messages. However, for security and management, it is always recommended to separate trusted and non-trusted items by creating separate BizTalk Hosts .

BizTalk Server 2004 natively supports two types of hosts. In-process hosts are those that the server can create and control. These hosts can receive and send messages and are used to process orchestrations. The second are isolated hosts , which are used to represent external processes like an ISAPI extension. These are created externally by adapters like the HTTP and SOAP adapter to host these external process types. When you view these as part of the tracking process, they are considered isolated and will always show as unknown because BizTalk is unable to access their current process state.

Each orchestration creates subscriptions to indicate the kinds of messages it wants to receive. When an appropriate message arrives in the MessageBox, that message is dispatched to its target orchestration, which takes whatever action the business process requires. The result of this processing is typically another message, produced by the business process and saved in the MessageBox.

This message, in turn , is processed by a send pipeline, which may convert it from the internal XML format used by BizTalk Server 2004 to the format required by its destination. The message is then sent out via a send adapter, which uses the appropriate mechanism to communicate with the application for which this message is destined.

For example, a business process may subscribe to Requisitions, which defines a certain message type, or the subscription may be more business oriented, as in a subscription for Capital Asset Purchases Greater than $10,000. Alternately, the subscription criteria may contain a location rather than a message type. For example, a business process may subscribe to all messages, regardless of type, that originate from a specific company.

Business processes are implemented as one or more orchestrations, each one consisting of executable code created using the .NET Framework. You don t create orchestrations by writing code; instead, a business analyst or developer graphically organizes a defined set of shapes to express the conditions, loops , and other behavior of a business process. Business processes can also use the Business Rules Framework, which provides a simpler interface to express business processes.

Business Rules Framework

The Business Rules Framework provides developers and designers with a mechanism for coupling business policies and logic with complex business processes. It combines an inference engine with a declarative model that allows the separation of implementation details from the business logic in rules. This allows you to update rules and policies that contain them without having to update code as you would in a traditional development cycle.

The Business Rules Composer, shown in Figure 7.4, is a graphical interface that allows developers, business analysts, and administrators to develop and apply policy-based rules. Many times, rule conditions are based on data sources that often have detailed, difficult-to-read binding information. This may tell the developer little or no information about what the data source actually refers to. The Business Rules Framework allows you to create vocabularies of rules that are tied to domain-specific terminology that can be associated with rule conditions and actions. Developers can bind business logic, change policies, and deploy these policies. Exposed as a set of functionality available within the Microsoft.RulesEngine namespace, they can be programmatically managed and edited.

Figure 7.4: The BizTalk Business Rules Composer.

When developing rules for use within an application, follow these steps, which show that the solution can be developed and deployed as shown in Figure 7.5.

1. Identify existing or new business logic that is to be represented as IF-THEN statements in the rules.

2. Identify the data facts that the business logic will be applied to. The rules engine can use data exposed through assemblies, classes, XML documents, or databases.

3. Create a policy and then define the rules to present the business logic. Associate the rule, condition, and action with the facts identified previously.

4. Test the policy to verify proper behavior.

5. Deploy the policy to a host application.

6. Create a policy inside an application or Web Service using the name of the defined policy as a parameter, as shown here:

    Dim xd as System.XML.XMLDocument     xd = incomingXMLMessage     xmldoc = new TypedXMLDocument("MyMessageType", xd)     myPolicy = new Microsoft.RuleEngine.Policy("MyPolicyName")     myPolicy.Execute(xmldoc)     myPolicy.Dispose()     OutgoingXMLMessage = xmldoc.Document 

7. Pass the data facts to the Policy object and execute the policy.

8. Review the results of the executed policy.

   
   Figure 7.5: A defined rule set within the Business Rules Composer.

XLANG

Orchestrations are used to design, execute, and manage a business processes. The logic necessary for this is maintained in an XML file format called an XLANG orchestration.

Within BizTalk Server 2004, it allows you to import and export orchestrations to the Business Process Execution Language for Web Services 1.1 (BPEL4WS) specification. The BPEL4WS language is a standard, platform-independent language for formally describing Web Service behaviors and processes. It was jointly developed by Microsoft and IBM to provide a standard mechanism for universal interoperability. It defines both business protocols and executable business processes in order to facilitate automation and integration of enterprise application and business-to- business processes, while at the same time improving their scalability.

BPEL4WS takes advantage of several XML specifications: WSDL 1.1, XML Schema 1.0, and XPATH 1.0. It employs WSDL messages and XML Schema type definitions for its data model, and XPATH for data manipulation. All external resources and partners are represented as WSDL services. BPEL4WS also provides extensibility to accommodate future versions of these standards.

BPEL4WS supports both synchronous and asynchronous peer-to-peer message exchanges within stateful, long-running interactions involving two or more parties. It also specifies precisely the mutually visible message exchange behavior of each of the parties involved in the protocol, without revealing internal implementations. In this way, businesses can retain privacy, and process implementations can be changed without affecting the public business protocol.

Since BPEL4WS clearly describes business protocols in a platform-independent manner and describes all cross-enterprise business behaviors, participants can understand and conform to a business protocol without having to work out ad hoc agreements, which have historically added much to the difficulty of establishing cross-enterprise automated business processes. It is important to keep in mind the conversion restrictions, shown in Table 7.2, when you re importing or exporting business processes.

Table 7.2: Conversion table for XLANG to BPEL4WS.

Type	XLANG/s	BPEL4WS
Literals	String, character Integer, real Boolean true , false Literal constant	XPATH string XPATH number XPath true(), false() functions XSD equivalent
Variables	Variable reference Message reference (.NET type) Message-part reference Distinguished-field reference Message data-property reference	bpws:getContainerData (%varName%, part, %locationPath%) bpws:getContainerData (%msgName%, part, %locationPath%) bpws:getContainerData (%msgName%, %locationPath%) bpws:getContainerData (%msgName%, %partName%, %locationPath%) bpws:getContainerProperty (%msgName%, %propertyQName%)
Operators	Unary + Unary - Unary ! Binary &&, Binary ==, !=, <=, <, >=, > Binary +, -, *, % with both integral operands	Ignored XPATH unary - Path not( ) function XPATH ˜and , ˜or operators XPATH ˜= , ˜! = , ˜<= , ˜< , ˜>= , ˜> operators XPATH ˜+ , ˜- ˜, ˜* , ˜mod operators

Additionally, the following XLANG constructs are disallowed in BPEL4WS:

• Message context-property reference

• Service-property reference

• Port-property reference

• Service link-property reference

• Unary ”with non-integral type

• Unary ~

• Cast operator

• Binary / with integral operands

• Binary +, -, *, %, / with non-integral operands

• Binary <=, <, >=, > with non-string operands

• Bitwise operators &, ^,

• Shift operators <<, >>

• Checked expression

• Intrinsic expression

• Pre- and post-increment and decrement ++, --

• Object invocation (with or without and/or ref parameters)

• new operator

XLANG has been a core component of BizTalk since the first product release but has been substantially enhanced within the new version. XLANG is now an extension of the managed runtime environment of the .NET Framework. This inclusion provides enhanced support for communication, transactions, long-running processes, and state management. These new managed orchestrations appear as a palette in Visual Studio.NET 2003, as shown in Figure 7.6. This palette allows developers to visually create, edit, and modify workflow processes directly within the development environment.

Figure 7.6: The XLANG components available within Visual Studio.NET 2003.

An additional extension to XLANG is a tracking engine. The Business Activity Monitor (BAM) allows business decision- makers to monitor business processes and track potential process bottlenecks. Using the dashboard interface of WSS, the business user can monitor, interact with, and even change existing BizTalk processes. The main feature of BAM is to provide a platform for continual process improvement that is easy to configure and manage.

Integration with the .NET Framework

The direct integration of BizTalk into the Visual Studio.NET 2003 IDE adds a new set of project types within Visual Studio.NET 2003, as shown in Figure 7.7.

Figure 7.7: Available BizTalk project types within Visual Studio.NET.

 CD-ROM      For example, if we created an orchestration that converted patient registration information collected within InfoPath to update a back-end mainframe, we would start within an empty BizTalk project. (This can be found in \Code\Chapter 7\PatientOrchestration\NewPatientOrchestration.xln on the companion CD-ROM.) This project type creates a blank project with default references to managed assemblies needed to create an orchestration. These default assembly references include the following:

• Microsoft.BizTalk.DefaultPipelines

• Microsoft.BizTalk.GlobalPropertySchemas

• System

• System.Xml

The Microsoft.BizTalk.DefaultPipelines is a reference to the BizTalk Pipelines module. The reference defines a .NET or COM component that links to the various processing stages of either the sending or receiving message. Each portion of these components specifies a messaging stage. For example, these stages include encoding/decoding, assembly/disassembly, and encryption/decryption. The pipeline designer allows the developer to modify, change, or augment the specific order of the processing stages.

The Microsoft.BizTalk.DefaultPipelines namespace defines the default pipelines, which are XMLReceive , PassThruReceive , XMLTransmit , and PassThruTransmit . The default pipeline reference can process documents using the PassThruReceive and PassThruTransmit function. The default processing order for the BizTalk Pipeline is shown in Table 7.3.

Table 7.3: Default pipeline processing order.

Pipeline	Order
XML Receive Pipeline	Decrypter Decoder Disassembler Validator Party Resolution
XML Transmit Pipeline	Assembler Encoder Encrypter

Defining Messages

Pipelines are used to process messages that are defined as an XML Schema (XSD). These schema definitions define and maintain the specific message formats and structures that are used during the orchestration. You add XSDs within Visual Studio.NET by using the Add New Item, as shown in Figure 7.8.

Figure 7.8: Adding a new schema item.

The schema as shown in Figure 7.9 represents the XML data that is being submitted by the InfoPath form. This schema represents the data that is saved as part of the form and defined within the InfoPath solution. The problem is that the data needed to post to the back-end mainframe system is actually a different format, as shown in Figure 7.10.

Figure 7.9: The InfoPath inbound schema definition.

Figure 7.10: The mainframe-required schema definition.

Once both of the schemas are completed, it is important to review and validate both sources to ensure that there are no semantic problems with the XSDs. In many ways, reviewing and comparing a specific message instance is a lot easier than reviewing the schema tree and data types. Creating XSD can be a very complex process that includes describing all the possible formats and variations within a specific document type. Visual Studio.NET allows you to generate an instance of the XML document to review the specific format structure. Within the Solutions Explorer, right-clicking on the schema and selecting Generate Instance creates a reference document for schema definitions.

Note  

Generating a default XML instance is not a foolproof way of validating a schema, but it will help you quickly validate the XML document for accuracy and data type inconsistencies.

Orchestration Design

Messages are the basic unit of communication within an orchestration. Messages consist of one or more parts with context data that describes their properties and content. BizTalk orchestration uses messages to provide the context for interchange between business process participants. Typically, orchestration is broader in scope than a traditional workflow.

Each participant is autonomous, and the responsibility of routing a work item is determined by each cooperating participant. BizTalk orchestration extends the definition of the traditional interaction diagram to include definition and control for decisions, concurrent actions, transactions, and supporting actions. The end result of a completed orchestration is an XLANG executable file that represents the description of the business processes.

Within the Visual Studio.NET 2003 IDE, new orchestrations are added in the same way as all other BizTalk items ”through the Solutions explorer and the New Item menu. This adds a file with an .ODX extension that contains the XLANG structures. Within the IDE, the orchestration page provides a visual interface that presents the underlying XLANG structure.

This interface is broken into three sections that define an orchestration, as shown in Figure 7.11. First is the actual design surface or palette. This area is the visual surface where toolbox items are dropped and configured to define the process flow. Using this surface, developers can drop the send and receive ports that define the inbound and outbound message handler. The second is the orchestration designer, which uses the Orchestration Types window to define the specific port types, correlation types, and role links that are available for use within the orchestration.

Figure 7.11: The BizTalk Orchestration palette.

Using this window, we can define the specific inbound and outbound operations defined for the orchestration. Finally comes the Orchestration Variable window, which shows the orchestration properties, parameters, ports, messages, variables, correlation sets, and role links available for the orchestration. This window is used to define and correlate the specific message to either the inbound or outbound port.

Mapping Schema Definitions

Within the orchestration designer, the transform shape enables the flow of data from one message to another. This can be done through a direct assignment or a mapping process. The NewPatientOrchestration required a specific map because of the required transformation and data translation to the outbound message type. The transform shape provides access to the BizTalk Mapper, which allows the assignment of specific source and destination attributes, and provides both transformation and translation of specific schema elements and attributes.

You initialize and define maps by placing the transform shape onto the design palette, as shown in Figure 7.12, and then defining the schema for the source and destination documents. This configuration is then loaded into the BizTalk Mapper, where the actual element and attribute flow from one message to another is defined.

Figure 7.12: The transformation object for the NewPatientOrchestration .

Within a map, functoids are used to manipulate the data flow. Functoids are an important part of any map that allow you to map elements and attribute data into different elements and attributes across different structures. Within BizTalk Server 2004, they have become part of the Visual Studio.NET 2003 toolbox palette and appear as part of the data map. Functoids are divided into the categories shown in Table 7.4.

Table 7.4: Categories of functoids available for mapping.

Functoid Category	Description
Advanced Functoids Reference	Use the Advanced functoids to configure various types of data manipulation, such as implementing custom script, value mapping, and managing and extracting data from looping records.
Conversion Functoids Reference	Use the Conversion functoids to returns a hexadecimal value when given a decimal value. This closely matches engineering functions such as DEC2HEX. Conversion functoids can also be used to convert a character to its ASCII value or a value to the corresponding ASCII character.
Cumulative Functoids Reference	Use the Cumulative functoids to perform various types of accumulation operations for values that occur multiple times within an instance message.
Database Functoids Reference	Use the Database functoids to extract data from a database.
Date and Time Functoids Reference	Use the Date and Time functoids to add date, time, date and time, or add days to a specified date, in output data.
Logical Functoids Reference	Use the Logical functoids to either perform specific logical tests at runtime or to determine whether output instance data is created at runtime.
Mathematical Functoids Reference	Use the Mathematical functoids to perform calculations by using specific values, called arguments , in a particular order, or structure.
Scientific Functoids Reference	Use the Scientific functoids to convert a numeric value to a scientific value. For example, the Cosine functoid takes a value in radians from a field or record and returns the value of the cosine.
String Functoids Reference	Use the String functoids to manipulate data strings by using string functions. For example, the String Find functoid finds one text string within another text string, and returns the position of the first character of the found string. Similarly, the String Concatenate functoid combines two or more input parameters into a single output field.

When working on a map, you can drag functoids directly onto the design palette. For example within the NewPatientOrchestration we dragged the string concatenation onto the map and dragged the source and destination elements between the two, as shown in Figure 7.13. Within a map, input links correspond to input parameters and lead to a functoid from the left, and an output link corresponds to the output parameters and leaves the functoid from the right.

Figure 7.13: Functoids and mapping within BizTalk.

Deploying the Solution

Once the map and orchestration are completed, the solution is ready for deployment. All projects within BizTalk Server 2004 are deployed into a managed assembly as a DLL. This is done by building the solution and then deploying it using the E-Business Web Services Wizard, as shown in Figure 7.14.

Figure 7.14: Publishing the completed BizTalk orchestration.

This wizard provides the option of deploying either an orchestration or enterprise schema as a Web Service. When deploying an orchestration, the wizard starts with the compiled DLL and generates the necessary namespace, Web Service name, communication patterns, and request/response objects to build and deploy an ASP.NET Web Service Project. The wizard identifies the defined ports, location, and schema within the compiled assembly to generate the necessary WSDL file that provides the Web Service definition.

Any orchestration that uses only a single Web port is published as a single ASMX file that defaults to the orchestration name. The NewPatientOrchestration used more than one Web port, and the default naming context is the name of the orchestration with an underscore followed by the Web port name. For example, once the NewPatientOrchestration was deployed, the Web port names NewPatientOrchestration_Outbound.asmx and NewPatientOrchestration_Inbound.asmx were assigned.

These are the names that InfoPath will use to bind to the orchestration using the InfoPath Web Service adapter, as shown in Figure 7.15. Once InfoPath is bound to the specific Web Service, InfoPath can submit and receive data from these defined BizTalk Services.

Figure 7.15: Binding the InfoPath form to the Web Service.