Appendix C: Emerging XML Standards

We are going to look at three interesting XML messaging frameworks in detail: SOAP, BizTalk, and ebXML. There are many others, but these have a number of the important characteristics that message wire formats need. They are also interesting to contrast because they try to meet different goals.

Messaging Frameworks

SOAP simply tries to define a basic XML format oriented toward the RPC-style of communications. It is both transport- and platform-neutral. SOAP is deliberately simple so that more complex features, such as those required in business messaging, can be incorporated into it.

Essentially, this is what BizTalk does: it extends SOAP, adding hooks for reliable messaging, service naming, outside transport bindings, etc.

Finally, ebXML looks at business messaging as a whole, including modeling and articulation of the processes behind message exchange. Out of this standard also comes an XML message format that meets the demands of a global market without reliance on a particular transport.

We have also included a brief look at XMSG, published in a W3C Note, and interesting for its straightforward approach some messaging challenges, and some coverage of the W3C's XML Protocol Activity.

Important 

A word of caution: this is a very fast moving area, and the information presented here is subject to a lot of change.

SOAP

The Simple Object Access Protocol (SOAP) is an XML messaging protocol co-authored by representatives from Microsoft, DevelopMentor, UserLand, IBM, and Lotus and submitted to the W3C. It provides a very simple and straightforward XML framework for defining messages. Of the messages formats we will examine, SOAP has so far generated the most interest from vendors and developers alike. You can download the SOAP specification from http://www.w3.org/TR/SOAP.

Although SOAP is intended to be the wire protocol for a distributed object system, there are a number of basic requirements that it explicitly does not address. These include distributed garbage collection and object references, both of which greatly add to application server complexity.

Similarly, the spec does not attempt to define object lifecycle functions, such as construction, activation, destruction, etc. SOAP does not attempt to define how to batch multiple messages together, a process called "Boxcarring". The spec does not preclude any of these things: a SOAP RPC may reference a remote, stateful object if the particular implementation provides this functionality; it just does not try to state how this might be done.

There is a danger here that we will be inundated with proprietary vendor implementations that do not interoperate (remember CORBA?). Nevertheless, there is considerable excitement around SOAP, and Microsoft has announced that it will be the foundation of its .NET platform.

Message Format

The SOAP message format is interesting to contrast against that of ebXML. The differences stem for the very different objectives of the groups that designed the messages. SOAP really is intended only as a simple and flexible message protocol. Its strength is that it does not try to over define; instead, it provides extensibility so application designers can add the additional functions they need.

Reliable messaging is a good example of SOAP's extensibility. SOAP does not implement reliability; however, Microsoft's BizTalk framework, described below, uses SOAP messages and adds the necessary fields to implement this (as well as a number of other functions). Because of this simplicity, SOAP is a good foundation to base new messages on when some of these features are delegated to infrastructure, such as the case in a number of JMS applications using robust MOM as a service provider:

Messages in SOAP are fundamentally one-way (the send-and-forget model). However, there are semantics for synchronous request/response (that is, the RPC model) defined in the spec, and this will likely be the dominant SOAP interaction model.

SOAP messages must be well formed, but not necessarily validated XML. In contrast to ebXML, described below, SOAP packaging makes use of XML. The spec defines a mandatory, namespace-qualified root element called an envelope, which is a container for both header and body elements:

     <?xml version="1.0" encoding="UTF-8"?>     <SOAP-ENV:Envelope     xmlns : SOAP-ENV="http ://schemas.xmlsoap.org/soap/envelope/ "     SOAP-ENV:encodingStyle="http: //schemas .xmlsoap. org/soap/encoding/ "/>     <SOAP-ENV:Header>     </SOAP-ENV:Header>     <SOAP-ENV:Body>     </SOAP-ENV:Body>     </SOAP-ENV:Envelope> 

The drawback of this is the difficulty in transporting anything that might break an XML parser handling the message. Binary attachments, invalid XML fragments, or even valid XML documents that contain their own root will cause problems. Typically, people will address this by escaping data in CDATA sections, escaping with parameter entities, or encoding with Base64.

Note 

Multiple attachments remain a problem; however, there is a W3C note exploring this issue. SOAP Messages with Attachments, released in December 2000, can be downloaded from http://www.w3.org/TR/2000/NOTE-SOAP-attachments-20001211. The note proposed a packaging using MIME multipart/related structure, which is the solution adopted by the ebXML committee.

The SOAP envelope contains an optional, namespace-qualified header. SOAP defines a number of header elements; however, the intent of the header scheme is to be entirely flexible, allowing users to add their own. This is one of the most powerful idioms in XML messaging. Header extension does not require that prior notice be served to a receiving party; the receiver will not consider additional, unrecognized headers an error condition. Header elements can contain two optional attributes:

     <SOAP-ENV:Header>       <n:UserDefHeader xmlns:n="..."Actor="..." MustUnderstand="...">         Header content       </n:UserDefHeader>     </SOAP-ENV:Header> 

The actor attribute identifies the application for which the header is intended. Note that SOAP messages can transit intermediaries (that is, they can be multi-hop). Intermediaries must remove headers targeted for them and process the message accordingly before forwarding.

The mustUnderstand attribute indicates whether the actor it identifies must process the header element. This is an interesting requirement brought about because of the flexibility of XML. On one hand, we promote XML as being good for messaging because minor changes to messages rarely break handlers; indeed, most handlers will simply ignore added elements or attributes.

Consider the case, though, where this behavior could lead to problems. What if you wanted to weed out any handlers that had not been updated to accommodate the message changes? This is the problem solved by the mustUnderstand attribute. If this attribute is set to "1", but the handler cannot process the header; it must signal an error.

The body is a mandatory, namespace-qualified container for further XML data. These data items are called body entries. Body entries may be further namespace qualified to disambiguate between schemas. The spec defines a fault element for communicating server problems back to a client. Applications render faults into the body section. There can only be a single Fault element present in the body:

     <SOAP-ENV:Fault>       <faultcode> "Fault code" </faultcode>       <faultstring> "String describing fault" </faultstring>       <faultactor> "Some URI" </faultactor>       <detail>           ...       </detail>     </SOAP-ENV:Fault> 

Four general faultcode categories are defined in the spec, categorizing faults as resulting from problems with the initial namespace (which defines SOAP version), problems with the message processor not understanding a part of the message it should, client errors (which include problems with message contents), and server errors (generally processing errors unrelated to the message content).

It is intended that faultcodes be extensible. The faultstring is intended for human consumption, not machine processing. Both the faultcode and the faultstring are mandatory. faultactor is an optional element containing a URL specifying the process where the fault was registered. This is important for messages that traverse multiple intermediaries. Finally, if any problem occurs in processing the contents of a message body, the resulting fault element must contain a detail element. The detail element is a container for detail entries, which further describe the error. Detail entry structure is flexible.

SOAP adopts primitive type definitions (int, float, string, etc) from XML Schema. The spec is quite detailed on how to render types, including compound types like arrays and structures, which greatly aids in creating deterministic bindings to other languages. For example, consider this simple Java code sample:

     public class FamousAuthors {       String[] author = {"Mark Twain", "Charles Dickens", "George Orwell"};     }     FamousAuthors myWriterList = new FamousAuthors(); 

This could be marshaled into the following document fragment using the SOAP conventions:

     <myWriterList>     <author SOAP-ENC:arrayType="xsd:String[3]">       <name>Mark Twain</name>       <name>Charles Dickens</name>       <name>George Orwell</name>     </author>     </myWriterList> 

In this case the array is defined in a schema as follows:

     <element name="author" type="SOAP-ENC:Array"/> 

If a schema is not defined, the above serialization is decomposed into:

     <myWriterList>       <SOAP-ENC:Array SOAP-ENC:arrayType="xsd:string[3]">         <SOAP-ENC:string>Mark Twain</SOAP-ENC:string>         <SOAP-ENC:string>Charles Dickens</SOAP-ENC:string>         <SOAP-ENC:string>George Orwell</SOAP-ENC:string>       </SOAP-ENC:Array>     </myWriterList> 

Similarly, consider a more complex example:

     public class Portfolio {       public String symbol;       public double value;       public Portfolio (String symbol, double value) {         this. symbol = symbol;         this.value = value;       }     }     Portfolio [] myPortfolio = new Portfolio[2];     myPortfolio[0]=new Portfolio ("WROX", 123 .45);     myPortfolio[1]=new Portfolio ("ABC",987.65); 

This would be marshaled in a SOAP message as:

     <SOAP-ENC:Array SOAP-     ENC: arrayType= "PortfolioTypeDefSchemaNameSpace:myPortfolio[2]">       <myPortfolio>         <symbol>WROX</symbol>         <value>123.45</value>       </myPortfolio>       <myPortfolio>         <symbol>ABC</symbol>           <value>987.65</value>       </myPortfolio>     </SOAP-ENC:Array> 

Again, the array myPortfolio must be defined in a schema that is referenced by the namespace PortfolioTypeDefSchemaNameSpace. Interested readers should refer to the SOAP specification for more details.

Transport Binding

A fundamental design goal of SOAP has been to stay simple and flexible, and make use of existing, proven technology, particularly transport infrastructure. The SOAP authors felt that this would actually make it more successful, as SOAP can readily adapt to the wildly varying requirements of different vertical markets. Version 0.9 of the SOAP specification actually explicitly tied it to HTTP. The idea was to create a protocol that could make extensive use of existing infrastructure (and there was a lot of this around). be able to easily traverse firewalls (since most are port 80/443 friendly), and take advantage of the mature HTTP security model.

To ensure that SOAP calls were upfront about their intent - after all, this is an API for remote processing, which will make most security managers nervous - all calls made use of the M-POST method, something provided by the HTTP extensions framework. By SOAP 1.1, the proposal had decoupled from a specific transport binding and made M-POST optional, acknowledging its lack of native support; however, HTTP is still the reference in the specification document, though one could equally use SMTP/POP3/IMAP4, or MOM systems.

Moving to a robust MOM system would provide reliable messaging features, asynchronous messaging, different messaging paradigms like Publish/Subscribe, etc. None of these is accounted for in a SOAP binding to HTTP (although the hooks can be built in, as we will see when we explore BizTalk).

IBM is working on a binding of SOAP to its flagship MOM product, MQSeries. A preview of this technology is available in the IBM Web Services Toolkit, downloadable from http://alphaworks.ibm.com/tech/webservicestoolkit. MQSeries, of course, provides a JMS API; however, this demonstration makes use of the product's proprietary Java API, based on the cross-language MQI specification. Nevertheless, it does serve as an inspiration and a clear indication that SOAP bound to MOM is coming and will be significant in the future.

What if you want to use SOAP as a messaging framework in a JMS application as of now? At present, there are no standard guidelines available that define a JMS binding; however, there are enough clues in the existing HTTP binding to be able to do this in such a way that we will likely align with future standards. The following is a simple SOAP request and response sequence, bound to HTTP. First the SOAP request:

     POST /soaptest/stocks/listeners/services.asp HTTP/1.0     User-Agent: Mozilla/4.0 (compatible; MSIE 4.01; MS ROPE Engine)     Accept:*/*     Authorization: dXNlcm5hbWU6cGFzc3dvcmQ=     Content-Type: text/xml     Content-Length: 455     SOAPAction:     http://palm.infowave.com/soaptest/stocks/listeners/services.asp#GetStockQuote     Host: palm.infowave.com     <?xml version='1.0'?>     <SOAP-ENV:Envelope        xmlns:SOAP-ENV='http://schemas.xmlsoap.org/soap/envelope/'        SOAP-ENV:encodingStyle='http://schemas.xmlsoap.org/soap/encoding/'        xmlns:xsi='http://www.w3.org/1999/XMLSchema-instance'        xmlns:xsd='http://www.w3.org/1999/XMLSchema-'>       <SOAP-ENV:Body>         <GetStockQuote xmlns='http://palm.infowave.com'>           <Symbol>WROX</Symbol>         </GetStockQuote>       </SOAP-ENV:Body>     </SOAP-ENV:Envelope> 

The SOAP response:

     HTTP/1.1 200 OK     Server: Microsoft-IIS/4.0     Date: Tue, 06 Feb 2001 18:59:15 GMT     MessageType: CallResponse     Content-Type: text/xml     Expires: Tue, 06 Feb 2001 18:59:15 GMT     Set-Cookie: ASPSESSIONIDGQGQQIGP=BINNEHKDOFDDDJKJFMKNKKFM; path=/     Cache-control: private     <?xml version='1.0'?>     <SOAP-ENV:Envelope        xmlns:SOAP-ENV='http://schemas.xmlsoap.org/soap/envelope/'        SOAP-ENV:encodingStyle='http://schemas.xmlsoap.org/soap/encoding/'        xmlns:xsi='http://www.w3.org/1999/XMLSchema-instance'        xmlns:xsd='http://www.w3.org/1999/XMLSchema'>       <SOAP-ENV:Body>          <GetStockQuoteResult xmlns='http://palm.infowave.com/'>            <result>123 .45</result>          </GetStockQuoteResult>       </SOAP-ENV:Body>     </SOAP-ENV:Envelope> 

SOAP request messages are contained as data of the HTTP POST method (or alternatively, M-POST for implementations that support it). The HTTP content-type parameter must be set to text/xml. InJMS, using the javax.jms.TextMessage class can imply this (theJMS spec is quite clear about the intentions of the TextMessage class). This should be further qualified using a custom JMS property defining the text as XML (as opposed to unstructured text).

SOAP has also defined a new HTTP header field: SOAPAction.SOAPAction is a URI indicating the purpose of the message. Usually, this would indicate object and method names in an RPC-style application. Servers will use this field to map the message to an appropriate handler. Pulling this out of the message and placing it into the transport binding is an interesting design decision.

The implication is that message handlers - such as a server, but also any intermediate - only need to be able to parse HTTP, not XML. This makes sense for a number of reasons. Parsing the simple, nonnested attribute:value header items of HTTP is much more efficient than XML; this makes operations like dispatch and routing very efficient.

There is also a wealth of existing Internet infrastructure that is already able to process HTTP headers. For example, the presence of SOAPAction as an HTTP header simplifies the task of building service-based filtering rules into firewalls, which may not have XML parsers, but are likely to be HTTP aware. A farm of servers providing web services could be physically partitioned using HTTP load balancers that distribute requests based on the service identified in the SOAPAction field.

Note that the URI of the initial HTTP request may also indicate the intent of the message. If this is the case, then set the SOAPAction value to an empty string.

SOAPAction is an obvious candidate for a JMS message property. This can also be a reasonable substitute for the lack of an easily parameterized URI in JMS applications. Once it is a property, the richness of the JMS message selection features become available to build extremely adaptable applications.

The properties functionality of JMS does provide us with an easy way of mapping the HTTP SOAP header entries. It also means we can leverage its powerful selection features. JMS in general also gives us most of the features we need to implement business messages, such as reliability, unique message identification, and message correlation IDs for request/response messaging over a shared channel. Making use of these does tie us to JMS, though. In the next sections, we will see how some of these features can be integrated directly into the XML messaging framework.

BizTalk

The BizTalk Framework is an XML messaging format defined by Microsoft to support business-to-business information exchange. Naturally, it integrates with the company's BizTalk server; however, the framework is intended to support integration between different systems and so is not coupled to this.

BizTalk is particularly intriguing because it shows how a very simple specification like SOAP can be extended to meet the requirements of business-to-business electronic commerce. BizTalk SOAP features reliable messaging, more sophisticated addressing that targets business processes, and transfer of complex data sets.

Like the ebXML committee, the BizTalk designers recognize the need for exchange of non-XML data, so the framework supports attachments using MIME conventions and the multipart/related content type. The MIME root element contains the SOAP envelope. The BizTalk specification calls this the "primary document."

The adoption of MIME, however, permits other data, including other valid or invalid XML documents, to be included as attachments, thus solving a persistent SOAP problem. MIME encoding also opens the possibility of using the secure version of MIME, called S/MIME, to secure attachments, assuming that the transport does not provide security on its own.

Note 

S/MIME provides security services on top of the original MIME specifications. It supports encryption and digital signing of messages. S/MIME is defined in RFC 2311, available at http://www.ietf.org/rfc/rfc2311.txt.

Message Format

BizTalk starts with the SOAP 1.1 specification, and then adds additional headers - called BizTags - to implement reliable messaging, routing, and many of the more generalized fields needed to exchange data in a business environment. There is also a diverse set of defined bindings, including HTTP, SMTP, FTP, MSMQ, DCOM, etc. At present, the specification is at version 2.0, which can be downloaded from http://www.microsoft.com/biztalk/techinfo/framework20.htm.

BizTalk introduces two new mandatory header elements to SOAP 1.1 (most namespaces have been removed for clarity - in an actual implementation, these must be included):

     <SOAP-ENV:Header>       <endpoints. SOAP-ENV:mustUnderstand="1">         <to>           <address xsi:type="..."> "Unique business entity ID"           </address>         </to>         <from>           <address xsi:type="..."> "Unique business entity ID"           </address>         </from>       </endpoints>       <properties SOAP-ENV:mustUnderstand="1">         <identity> "Globally unique message ID" </identity>         <sentAt> "Date time" </sentAt>         <expiresAt> "Date time" </expiresAt>         <topic> "Some URI" </topic>       </properties>     </SOAP-ENV:Header> 

The endpoints element identifies the business parties involved in the information document exchange. Note that any application processing this message must process this header entry; it is an error to ignore it, as signified by the value of the mustUnderstand attribute. The address element could contain a URI, or alternatively some other unique identifier that both parties recognize. The xsi:type attribute must characterize this identifier (the xsi:type attribute was introduced with XML Schema as a way of identifying types in instance documents).

The inclusion of source and destination identifiers acknowledges the focus of BizTalk on application-to-application business transactions. Basic SOAP bound to HTTP only provides the SOAPAction parameter (or potentially the original POST URL) as a means of identifying the destination application.

BizTalk explicitly supports an addressing model decoupled from transport. Not only is it available (and required) in the header, but the schema puts few restrictions on its form: as long as the applications can understand it, it is valid. Note also that by providing source and destination addresses, dynamic routing of messages across intermediates, which is common in business interactions, becomes possible.

The identity element, subordinate to properties uniquely identifies this particular document instance. This must be globally unique, and is used to implement request/response semantics, message acknowledgements (described below), and for general logging purposes. The sentAt and expiresAt elements are necessary for reliable messaging and basic messaging application architecture. For example, most queueing applications will set expiries on messages simply to avoid swamping queues if problems arise with the message receivers.

In Pub/Sub applications, it is common for certain items to be timely (such as a stock price). The topic element supports application-level routing decisions, and is really a nod toward messaging paradigms like Pub/Sub.

BizTalk further defines three optional headers: services, manifest, and process. The first of these is as follows (again, namespaces are removed):

     <SOAP-ENV:Header>        <!-- Mandatory endpoints and properties elements removed for clarity -->       <services SOAP-ENV:must:Understand="1">         <deliveryReceiptRequest>           <sendTo>             <address> "Some URI" </address>           </sendTo>           <sendBy> "Date time" </sendBy>         </deliveryReceiptRequest>         <commitmentReceiptRequest>           <sendTo>             <address> "Some URI" </address>           </sendTo>           <sendBy> "Date time" </sendBy>         </commitmentReceiptRequest>       </services>     </SOAP-ENV:Header> 

BizTalk actually provides a very rich environment for specification of deadlines, allowing developers to bound transport and transaction times. This manifests in the deliveryReceiptRequest and commitmentReceiptRequest headers, each of which has individually specified deadlines, and the general message expiresAt element in the header.

DeliveryReceiptRequest is used provide functionality similar to the JMS Session.AUTO_ACKNOWLEDGE. When a recipient accepts the message, it must reply to the address by the stated deadline.

CommitmentReceiptRequest informs the handler that it must send an acknowledgement when it commits to processing the message, implying that it has parsed and understood the message contents. It is similar to the JMS javax.jms.Message.acknowledge() feature under the Session.CLIENT_ACKNOWLEDGE option.

The manifest header is used to indicate that this document has multiple related parts. The manifest provides URI references using the SOAP href attribute, either to additional information in the SOAP body (called the primary document), in an attachment encoded in a MIME part, or potentially to an external resource outside of the message:

     <SOAP-ENV:Header>        <!-- Mandatory endpoints and properties elements removed for clarity -->       <manifest SOAP-ENV:mustUnderstand="1">         <reference>           <attachment href="..."/>           <description> "String description" </description>         </reference>       </manifest>     </SOAP-ENV:Header> 

There can be multiple references in a manifest. The example shows a reference with an attachment tag; references can alternatively contain a document tag, which contains a reference to the primary document. If this reference is to an attachment, it uses a content ID URL (CID) scheme. Content ID URLs are described in RFC 2111 (see http://www.ietf.org/rfc/rfc2111.txt). This provides a URL that points to the MIME part's Content-ID header. The description element simply contains additional text describing the reference. This element is optional.

Finally, BizTalk includes an optional header that identifies the business process associated with this document. For example, two airline reservation systems might exchange messages concerning availability of seats on a particular flight segment at a specific time. The business process could be a SeatAvailability. The process header uses URI references to identify both the process and a particular instance of the process involved in the communication, as multiple instances could potentially be running and require unique identification to maintain stateful communications. It also provides a detail container that can hold additional application-specific XML data.

     <SOAP-ENV:Header>        <!-- Mandatory endpoints and properties elements removed for clarity -->       <process SOAP-ENV:mustUnderstand="1">         <type> "Some URI" </type>         <instance> "Some URI" </instance>         <detail>             ...         </detail>       </process>     </SOAP-ENV:Header> 

Note that instance provides the basis for providing server-side sessions, much as a cookie does on the Web. The detail element is flexible, and can contain application specific information.

Receipt headers can also be included as responses to a message containing the deliveryReceiptRequest or commitmentReceiptRequest elements:

     <SOAP-ENV:Header>        <!-- Mandatory endpoints and properties elements removed for clarity -->       <deliveryReceipt SOAP-ENV:mustUnderstand="1">         <receivedAt>"Date time" </receivedAt>         <identity> "Some URI" </identity>       </deliveryReceipt>       <commitmentReceipt SOAP-ENV:mustUnderstand="1">         <decidedAt> "Date time" </decidedAt>         <decision> "Positive or negative" </decision>         <identity> "Some URI" </identity>         <commitmentCode> "Specific code" </commitmentCode>         <commitmentDetail>               ...         </commitmentDetail>       </commitmentReceipt>     </SOAP-ENV:Header> 

Here, identity refers to the identity of the document that requests the receipt. Recall that this is a required element of every document, and can be found in the properties header described previously. The decision element is simply positive or negative, declaring whether the process committed or not. The commitmentCode field is a qualified name that describes the status more specifically than the simple positive or negative status of the decision element. Applications needing to communicate further details can use the commitmentDetail element to contain customized structures.

ebXML

The Electronic Business XML Initiative (ebXML) is a joint undertaking between UN/CEFACT (the United Nations Centre for Trade Facilitation and Electronic Business, see http://www.unece.org/cefact) and OASIS (the Organization for the Advancement of Structured Information Standards, see http://www.oasis-open.org).

UN/CEFACT describes itself as responsible for "worldwide policy and technical development in the area of trade facilitation and electronic business". OASIS is "a non-profit, international consortium dedicated solely to product-independent data and content interchange". The ebXML group formed in November 1999, budgeting an 18-month timeframe to define the framework necessary to enable a single, global electronic marketplace. Details on the project are available at http://www.ebxml/org.

Of course, electronic exchange of business data is nothing new: EDI has been available for years as a solution to streamline information exchange and eliminate the need for many business documents. However, the older EDI protocols such as EDIFACT and X12 share the problems of many traditional messaging frameworks: they are extremely complicated and tightly coupled; the systems that support it are expensive to procure and difficult to support. As a result, EDI has seen widespread use only by very large organizations. The Fortune 100 use EDI; the corner store does not.

ebXML has set out to change this. One of its primary goals has been to build a generalized business messaging framework that will accommodate not just the industry giants, but also address the particular needs of developing nations and small to medium-sized businesses. To achieve this, ebXML must be able to run securely over public networks, and make use of inexpensive, commodity technologies. It would also leverage XML as a framework for exchanging business data. The ebXML effort chose XML because of its flexibility, its internationalization capabilities (UTF-8 or UTF-16 are to be used as a character encoding), and the readily available supply of processing technology.

It is an ambitious goal. What makes ebXML particularly compelling is that it goes quite a bit farther than simply defining message formats. It actually defines an entire process for modeling business interactions. It even recommends UML as a common modeling language to document the business relationships between trading partners.

In ebXML, every trading partner is described by a profile. In addition to the typical information like name, address, etc. that one would expect, the profile contains information about services that the organization offers, and a list of its business processes. Business processes - which the committee calls "choreographies" - are a focal point of the ebXML work.

Choreographies describe the messaging dialogs (which really involve an exchange of business documents) necessary to produce a business transaction. Business processes also characterize relationships between organizations doing business, including defining the roles and responsibilities each assumes.

Core components - data such as name, address, etc., and including both atomic types and aggregations of these - have been defined in a syntactically neutral manner; this way, they can be implemented in a diverse assortment of languages and shared between clients and servers. The ebXML committee also intends to describe globally distributed repositories to contain the business processes and profiles of trading partners. Repositories include message schemas, along with the query semantics supporting interface discovery, loading of core business components, etc.

Due to the high profile and ambitious objectives of the group, a number of other independent development efforts have begun to participate. A recent addition to the organization is a working group on trading partner agreements, which is starting with IBM's tpaML as a basis.

The Open Travel Alliance (OTA), which is defining XML-based standards for exchange of travel industry information, has contributed its profile specification. The OTA has defined messages for CRUD operations on remote profile servers (where profiles contain data like name, address, telephone number, etc.). OTA messages are interesting because profile servers can maintain state, and can implement simple row locking functionality that is accounted for in the message. They also define security information in the XML message, including authentication credentials.

The ebXML group met in May 2000 in Brussels and demonstrated a proof-of-concept, in which a small company exchanged OTA-style messages with a large enterprise. Message transfer utilized both HTTP and SMTP as transport protocols. By their August meeting, they were able to offer a dynamic trading network between participating vendors, which demonstrated a number of typical transport models such as point-to-point, hub-and-spoke message brokering, and message distribution across federated servers. In December 2000, the committee announced that it was two months ahead of schedule, and would release a formal specification in March 2001.

Message Format

The ebXML specification as a whole is very detailed and comprehensive in its approach to messaging, and to do it justice would really require a book in itself (and no doubt there will be titles covering this soon). What is most relevant to this section, however, is the ebXML message structure, which should serve as a guide on how to construct generalized wire protocols using XML.

The ebXML Messaging Service Specification, published by the Transport, Routing, and Packaging group (see http://www.ebxml.org/project_teams/transport/transport.htm), describes message formats and their relationship to transport. The ebXML committee focused on keeping messages simple, yet still robust even when transported across potentially unreliable channels. It defines the basic functions needed to serve the needs of a global marketplace, but additionally leaves the door open for vendors to add additional fields to realize increased reliability and functionality.

The ebXML proposal is explicitly decoupled from any individual transport. The Overview and Requirements document for Transport, Routing and Packaging (available at the same link), still in draft at the time of writing, offers HTTP, SMTP, FTP, CORBA, and commercial MOMs as potential transports. Thus, JMS would be a likely API to access transport, and this is exactly what JAXM, described below, intends to facilitate.

However, to be an effective global electronic business protocol, the authors recognize that reliability and security are essential. Because they cannot simply delegate these to transport infrastructure (impossible when these qualities could not be expected from HTTP or SMTP), they are incorporated into the message. There are facilities - some optional, some not - to account for security, quality of service, message routing, audit trails, restart and recovery, etc.

Message headers can contain timestamps, may have maximum lifetime fields, response addresses, priority fields, references to external documents contained in a manifest, etc. There are extensive requirements for error reporting, and high expectations on acknowledgements. Digital signatures can be applied to payload documents and to message headers. There is also support for ACID transactions. Although most ebXML documents reside still in draft form, overall it is a startlingly comprehensive and well thought out piece of work:

All ebXML messages are contained in an outer envelope defined by the external communications protocol, such as HTTP. Within this container is a message envelope defined as a MIME multipart/related message. This is independent of the transport protocol. MIME was chosen over an XML-based packaging for a number of reasons. There was no widely accepted spec for packaging messages with XML when the committee formed (indeed, at the time of writing, there still is not). In contrast, MIME, although developed for SMTP, is widely accepted for packaging both SMTP and HTTP content - no changes to this specification would be required to support ebXML. In contrast, a strictly XML-based solution would have required all non-XML data to be encoded.

Thus, the spec acknowledges that not all data interactions lend themselves well to XML packaging. By encoding them as MIME messages, non-textual data such as binary picture formats can be encapsulated in well-supported formats, which is often preferable over attempting to embed piecemeal in documents.

In fact, ebXML opens its definition of document as simply "any data that can be represented in a digital form". This includes sets of XML elements, PDF files, any binary format, and document fragments. Encryption and signing was an important consideration. Processing efficiency may also have been a consideration. Transactional EDI implementations often have to process very high message volumes. A general criticism leveled at XML messaging is that parsing into DOM trees may be too processor intensive for some high performance applications, such as a stock exchange. MIME is not structurally complex, so lends itself to very efficient processing.

The message envelope described above contains one or two further structures, which are actually MIME headers followed by content. There is a mandatory header envelope, which further decomposes into an XML header document. There is also an optional payload envelope, which contains the actual payload document, which may or may not be in expressed in XML. The XML header document structure is as follows:

     <?xml version="1.0" encoding="UTF-8"?>     <ebXMLHeader         xmlns=http://www.ebxml.org/namespaces/messageHeader Version="0.8"         MessageType="...">       <Manifest>       ...       </Manifest>       <Header>       ...       </Header>       <Routing Header>       ...       </Routing Header>     </ebXMLHeader> 

The version number reflects the version available at the time of writing. The attribute MessageType must take on one of the following values:

• Normal

• Acknowledgement

• Error

These values indicate (not surprisingly) whether this is a normal message, and acknowledgement, or an error message. For example:

     <ebXMLHeader         xmlns=http://www.ebxml.org/namespaces/messageHeader Version="0.8"         MessageType="Normal"> 

The acknowledgement value is interesting, because it is the first indication that the design of ebXML implements reliable messaging over potentially unreliable transports. At present, the specification defines this only for directly connected applications (that is, they are not using visible intermediates). Future versions will implement reliable messaging across intermediates.

The manifest is a container for URL-based references to documents associated with the message (namespaces and attributes have been removed for clarity):

     <ebXMLHeader>       <Manifest>         <DocumentReference>           <DocumentLabel>             "Document description code"           </DocumentLabel>           <DocumentId> "Some URL" </DocumentId>         </DocumentReference>       </Manifest>     </ebXMLHeader> 

These associated documents may be contained within a MIME body part elsewhere in this message. They may also be external to the message.

The generic header element (which is a sub-element of the ebXMLHeader) contains a number of children:

     <ebXMLHeader>        <! -- Mandatory Manifest element removed for clarity -->       <Header>         <From>           <PartyId context="...">             "Some ID, possibly a URN"           </PartyId>         </From>         <To>           <PartyId context="...">             "Some ID, possibly a URN"           </PartyId>         </To>         <TPAInfo>         ...         </TPAInfo>         <MessageData>            ...         </MessageData>         <ReliableMessagingInfo DeliverySemantics="..."/>       </Header>     </ebXMLHeader> 

To and From elements obviously identify the communicating parties, using a subordinate <PartyId> element that can use a URN or other identifier that may be more appropriate for the operation. In contrast to BizTalk, this element does not have an xsi:type attribute explicitly identified in the specification, but the context attribute is intended to serve the same purpose. TPAInfo contains information describing the trading partner agreement. Trading partner agreements may document rules under which trading can occur, locations to report errors, etc. This element includes:

     <Header>        <! -- Mandatory From and To elements removed for clarity -->       <TPAInfo>         <TPAId> "Some URI" </TPAId>           <ConversationId> "Some URI" </ConversationId>           <ServiceInterface>             "Some service ID, possibly URN"           </ServiceInterface>           <Action>             "Some process ID within the service interface"           </Action>        </TPAInfo>     </Header> 

TPAId identifies the actual trading partner agreement. In the event of an error, this may be used to determine where to send an error message (or a specific error handler might be identified, as described below). ServiceInterface and Action ultimately name the service that is to process the message. It is interesting to note how deeply embedded in the structure these data reside, demanding significant parsing to extract. Services may also appear in the payload document. Most implementations will likely map ServiceInterface and Action to a business object and method.

The MessageData element in Header contains additional elements to implement the functions commonly provided by message-oriented middleware. JMS provides access to the service provider's equivalents of these through the javax.jms.Message class:

    <Header>       <!-- Mandatory From, To, and TPAInfo elements removed for clarity -->      <MessageData>        <MessageID> "Unique message ID" </MessageID>        <TimeStamp> "Date time" </TimeStamp>        <RefToMessageID>          "Reference to unique message ID"        </RefToMessageID>      </MessageData>    </Header> 

Note 

MessageID must uniquely identify a message, using the message identification rules outlined in RFC 2392, Content-ID and Message-ID Uniform Resource Locators, which can be found at http://www.ietf.org/rfc/rfc2392.txt.

RefToMessageId is a correlation identifier, so that request/response messaging can build on top of the ebXML message framework. The TimeStamp here must conform to the Coordinated Universal Time (UTC) format described in ISO-8601 (http://www.iso.ch/markete/8601.pdf). This representation uses the model CCYYMMDDThhmmss.sssZ where T is a time indicator, used here to delimit, and Z is a timezone indicator, also used here to delimit. This works with the XML Schema derived data types for date and time. For example:

    <MessageData>       <MessageID>mid:[e-mail]</MessageID>       <TimeStamp>20011231T235959.999Z</TimeStamp>       <RefToMessageID></RefToMessageID>    </MessageData> 

The ReliableMessagingInfo child of the Header element contains a single attribute, DeliverySemantics, which can be set to one of the following values:

• OnceAndOnlyOnce

• BestEffort

If set to the former, additional elements will be required in the message to implement reliable messaging independent of transport:

    <Header>       <!-- Mandatory From, To, TPAInfo, and MessageData elements         removed for clarity -->    <ReliableMessagingInfo DeliverySemantics="..."/>    </Header> 

The RoutingHeader contains the following subordinate elements:

    <ebXMLHeader>      <RoutingHeader>        <SenderURI> "Some URI" </SenderURI>        <ReceiverURI> "Some URI" </ReceiverURI>        <ErrorURI> "Some URI" </ErrorURI>        <TimeStamp> "Date time" </TimeStamp>        <SequenceNumber> "Integer sequence no." </SequenceNumber>      </RoutingHeader>    </ebXMLHeader> 

SenderURI defines actual message handlers, as opposed to a party identifier defined in the Header described above. Note the identification of a specific error handler. This is quite a bit more flexible than simply having an error flag rendered in the message and assuming that every receiver can process it.

The SequenceNumber is another field used to implement reliable messaging, and is only included if one time delivery has been indicated earlier. The specification describes the actual algorithm to implement and it is well worth reading. Even when working with JMS over a reliable service provider, it is valuable to understand what the service provider actually has to go through to ensure reliability, as this always has performance implications.

The ebXML spec also contains a detailed framework for error reporting. ebXML does not render errors in the header; instead, the MessageType attribute in the ebXMLHeader is set to "Error" and a dedicated XML error document is inserted into the payload. The error document structure is as follows:

    <?xml version="1.0" encoding="UTF-8"?>    <ebXMLError xmlns=http://www.ebxml.org/namespaces/errorVersion="0 . 8">      <ErrorHeader ID='...'>        ...      </ErrorHeader>      <ErrorLocation>      </ErrorLocation>    </ebXMLError> 

There can be only a single ErrorHeader, but zero or more ErrorLocation fields. The ErrorHeader contains up to four subordinate elements:

    <ebXMLError>      <ErrorHeader ID='...'>        <ErrorCode> "Error code string" </ErrorCode>        <Severity> "Warning or Error" </Severity>        <Description xml:lang='...'>          "String describing error"        </Description>        <SoftwareDetails>          "String describing software details"        </SoftwareDetails>      </ErrorHeader>    </ebXMLError> 

Note that it optionally provides for both localized language text description of the error (which could be output to a user), and more detailed, developer-level error messages. ErrorCode's include one of the following defined in the specification. For error in the header document:

• UnableToParse

• ValueNotRecognized

• NotSupported

• Inconsistent

• OtherXML

Or for problems that are not related to XML documents:

• MessageTooLarge

• MimeProblem

• Unknown

The actual error code should be one of the above with a narrative description appended to it. This should be localized when practical to make use of the language identified in the Description element. Severity can take one of two values:

• Warning

• Error

ErrorLocation identifies where the error occurred. Since it may be an error in the message itself, it can refer to a part of the message. Alternatively (or in addition multiple locations are permissible), it can refer to something outside the message:

    <ebXMLError>       <!-- Mandatory ErrorHeader element removed for clarity --       <ErrorLocation ID='...'>          <RefToMessageId> "Unique msg ID" </RefToMessageId>          <Href> "Reference to message" </Href>       </ErrorLocation>>    </ebXMLError> 

BizTalk vs. ebXML

It is interesting to compare BizTalk and ebXML. Looking only at message formats, they are really far more similar that they are different. Superficially, the tags have different names, but for the most part, there exist equivalent constructs in both. Furthermore:

• Both designs emphasize flexibility and extensibility; their use of XML as a header container makes it very simple to add new tags and structures to the basic header

• Both decouple from transport, and provide the key tags needed to maintain reliable messaging services

• Both have a flexible means to render exceptions and communicate them to other systems (here, BizTalk uses existing SOAP error reporting mechanisms)

• Neither explicitly supports transaction hooks, although this could be added easily within the frameworks

The BizTalk deliveryReceiptRequest and commitmentReceiptRequest headers are particularly detailed and flexible, and have no direct equivalents in ebXML. Furthermore, BizTalk provides rich semantics for message, delivery receipt, and commitment receipt expiries, all of which are very important, and have direct equivalents in JMS.

The ebXML standard explicitly supports a version model for the message framework. The ebXML RoutingHeader is nicely separated and distinct, acknowledging the importance of intermediates in global business messaging. The TPAInfo header nicely encapsulates the logical concept of business process, the articulation of which is an important goal of the ebXML group, and reflects its larger focus as a horizontal framework for global electronic commerce.

XMSG

XMSG is an XML message format described in a W3C Note published in Oct 2000, residing at http://www.w3.org/TR/xmsg. The goal of XMSG is to define a simple XML container that could be used to transport one or more XML documents, or other data rendered as MIME types. It provides a flexible means of adding metadata about the contained documents as property fields. XMSG is simple, yet still supports a number of the fields necessary to support more advanced messaging.

Unlike BizTalk and ebXML, which define these fields as elements in a header structure, XMSG declares these fields as attributes of the message envelope:

    <message xmlns="..."                 to="..."                 from="..."                                  generated.on="..."                 reply.to="..."                 originator="..."                 originator.                 priority="..."                 expires="..."                 tracking.code="..."                 action="..."                 manifest="..."                 receipt.required="..."                 for.receipt="...">       <hop received.on="..." received.by="..." transport.used="..."/>       <property name="..." value="..."/>       <document uri="..." version="...">          ...       </document>    </message> 

Note the hop element. XMSG explicitly records the message trail across intermediates, as SMTP does. The originator and the originator.id attributes record the original sender of a message and its original unique ID (the from and id attributes record these properties only for the current hop).

Properties are extensible, although the specification does explicitly offer a number of properties mirroring the attributes in the message element. The document element can contain either XML documents, non-XML data encoded as a MIME type, or a reference to external data.

The document element, depicted above, may be replaced by either a failure element or a receipt element. The failure element contains a text description of why the failure occurred:

    <failure on="..." reason.code="...">       "Description of cause of failure"    </failure> 

The receipt elements exist for applications requiring acknowledgements of message receipt:

    <receipt message.tracking.code="..." timestamp="..."/> 

XMSG provides a good, simple framework for messages. It is likely that some of these ideas will resurface in the W3C's XP initiative, described below.

W3C XP

The XML Protocol Activity was launched under the W3C to define a standardized XML messaging framework. The activity formally started in September 2000, allocating an 18-month timeframe and a goal to deliver a formal recommendation in the fall of 2001. The XML Protocol Working group will actually define an XML messaging framework. The group intends to incorporate a number of the qualities that made SOAP so popular, but at the same time address its limitations as a generalized messaging framework.

The Activity will attempt to keep the specification simple and very easy to extend. It will include a specification for a messaging envelope, and a specification for serialization of application data. XP (as it has come to be know) primarily targets RPC-style communications, and the framework will describe typical Request/Response message formats.

However, XP will not be limited to this, so it will be appropriate as a message format for applications using paradigms like Publish/Subscribe. XP explicitly decouples transport and message format (that is, everything contained in the XP envelope), although the specification will define a default binding to HTTP.

Outside of the message formatting, the Activity will define XML protocol modules, which provide the additional services needed in messaging applications. These could include security services, interaction with caches, triggering of applications and events, message routing, etc. It acknowledges that messaging goes beyond simple direct connections and will accommodate intermediates in message transfer. The Activity declared repositories and service discovery outside of its scope.

XP has the influence of the W3C behind it, a stellar roster of participants from the XML and messaging community, and the opportunity to address the problems in the current XML message offerings.

Note 

A working draft of the XML Protocol Requirements, published in December 2000, is available on the Activity web site at http://www.w3.org/2000/xp. It will be interesting to watch what happens to XP.

Others

The formats described in this section are certainly not the only ones available. The point of this section was not to review all the formats in existence, but rather to examine a few to illustrate how they solve some typical messaging problems. There are other formats, like RosettaNet, WDDX, RDF, and ICE that could equally have been included. For those interested in exploring other protocols, the W3C keeps an excellent matrix comparing a number of formats at http://www.w3.org/2000/03/29-XML-protocol-matrix.