8. Reliable Delivery of BizTalk Documents

[Previous] [Next]

BizTalk facilitates asynchronous document exchanges involved in e-commerce and Enterprise Application Integration, where specific delivery guarantees and error detection and reporting are necessary for integration of business functions across domain boundaries. High-performance-messaging middleware solutions for the Internet are emerging and should be used for this purpose when available. However, given the broad scope of deployment scenarios for BizTalk Framework-based application integration, and the continued use of transports with lower guarantees of service, it is important to provide a simple standard solution for reliable delivery of BizTalk Documents that can be easily implemented by BFC servers. This solution is described in the current section and is based on two simple notions:

• Document receipts.
• Idempotent delivery.

The overall purpose is to ensure a defined outcome for BizTalk Document delivery. The following points summarize the ideas on which the reliable delivery functionality described here is based:

• Receipts provide a way for (the BFC server at) the source business entity to assure itself that the BizTalk Document was received and accepted, using the identity for correlation.
• Given the possibility of multiple transmissions of the same BizTalk Document due to retry, (the BFC server at) the destination business entity may apply idempotent delivery rules to detect and eliminate duplicate Documents, again using the identity for correlation.
• If (the BFC server at) the source business entity does not receive a delivery receipt within the timeout period specified in the receiptRequiredBy element, a delivery failure report will be generated and corrective action taken.
• Note that there is a small but finite possibility that the Document will be delivered to the destination business entity and a delivery failure report will nevertheless be delivered to the source business entity, since all receipts may be lost due to transport failure.

The rest of this section describes the structure of a receipt document and the typical behavior of the source and destination business entities engaged in reliable delivery of BizTalk Documents. The details of actual behavior are implementation dependent and the description here is meant to serve as a guideline.

8.1 Structure and Content of Receipts

A BizTalk Framework Receipt is a SOAP 1.1 message that contains at least the <receipt> and <properties> header entries, and an empty body, as shown in the example below. A receipt is not a BizTalk Document. It is a SOAP message that acknowledges receipt of a BizTalk Document.

The content of the receipt unambiguously identifies the BizTalk Document being acknowledged, by including the universally unique identity of the Document. A timestamp for the time at which the Document was received at the destination is provided in the <receipt> header entry.

The following receipt corresponds to the initial example in this specification:

 <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"> <SOAP-ENV:Header> <rct:receipt SOAP-ENV:mustUnderstand="1" xmlns:rct="http://schemas.biztalk.org/btf-2-0/receipt"> <rct:receivedAt>2000-05-15T04:08:10-05:30</rct:receivedAt> </rct:receipt> <prop:properties SOAP-ENV:mustUnderstand="1" xmlns:prop="http://schemas.biztalk.org/btf-2-0/properties"> <prop:identity> uuid:74b9f5d0-33fb-4a81-b02b-5b760641c1d6</prop:identity> <prop:sentAt>2000-05-14T03:00:00+08:00</prop:sentAt> <prop:expiresAt>2000-05-15T04:00:00+08:00</prop:expiresAt> <prop:topic>http://electrocommerce.org/purchase_order/</prop:topic> </prop:properties> </SOAP-ENV:Header> <SOAP-ENV:Body/> </SOAP-ENV:Envelope> 

In addition to the <receipt> header entry, a receipt must contain the <properties> header entry from the Document being acknowledged.

Understanding and processing of the <receipt> header entry and all its contents at the recipient BFC server is always mandatory during successful processing of a receipt. The encoding of the <receipt> element must always contain the SOAP-ENV:mustUnderstand="1" attribute to reflect this. The following table lists the BizTags used to construct the subelements of the <receipt> header entry and their properties:

Tag Name	Mandatory	Kind	Type	Occurs
receivedAt	yes	element	timeInstant	once

The <receivedAt> tag is the receiving timestamp for the Document acknowledged by this receipt. In the case of multiple copies of the BizTalk Document being received and accepted (see below for a discussion of the term "accepted"), the receiving timestamp may reflect either the time at which the first copy was received or the time at which the copy being acknowledged was received.

8.1.1 Behavior of the Source Business Entity

In accordance with common practice, it is assumed that the BizTalk Document being transmitted has been persisted in durable storage at the source business entity. It is strongly recommended that persistence in durable storage occur before a BizTalk Document is transmitted in the case of Documents that require reliable delivery with a defined outcome. However, this specification definitively specifies only wire-level behavior. Actual behavior at a business entity regarding storage and archival of business documents or the management of durable storage resources is implementation dependent and does not affect compliance with this specification.

The only special behavior required from a source business entity for reliable delivery of a BizTalk Document is to add a <reliability> element to the <delivery> header entry and to:

• Set the value of the <sendReceiptTo> subelement to an appropriate transport URL for receiving the receipt.
• Set the value of the <receiptRequiredBy> subelement to the required deadline for receiving the receipt.

The following description is a guideline for heuristic retry behavior that is likely to increase the probability of successful delivery in the presence of unreliable transports. The only required behavior in the context of retries is that the content of the BizTalk Document including header entries must not be altered in any way for a retry. In particular, the content of the <sentAt> property must remain the same, that is, what it was set to for the first transmission attempt.

The retry behavior of the source business entity is typically based on a parameter: the retry interval. This parameter may be fixed or may be configurable for each business relationship or even each transport used in the context of a given relationship. There is typically also a maximum retry count. The basic behavior pattern is very simple. Keep transmitting the document to the destination business entity at a frequency determined by the retry interval until one of the following occurs:

• A receipt is received.
• The receiptRequiredBy deadline expires.
• The maximum retry count is exceeded.

At the end of this process, if a receipt has not been received, and the receiptRequiredBy deadline expires, the delivery of the BizTalk Document is said to have failed. Normal operating procedure if this occurs is to notify the source application in an appropriate way, for instance by placing a copy of the Document in a dead-letter queue, but this is clearly implementation dependent.

It is worth noting that retries need not occur over the same transport that was used for the first transmission attempt. If multiple transport endpoints are available for transmission to the destination business entity as specified in the <to> address in the <delivery> header entry, new transport endpoints may be tried during retry attempts.

Actual implementations may use more elaborate algorithms for scalability and efficiency of resource use, as well as to account for peculiarities of the implementation context, such as intermittent connectivity.

8.1.2 Behavior of the Destination Business Entity

In accordance with common practice, it is assumed that, upon being accepted, each BizTalk Document will be persisted in durable storage at the destination business entity. It is strongly recommended that persistence in durable storage occur before a receipt is sent in the case of reliable delivery. In addition, idempotent delivery requires a minimum duration of archival for some information as noted below. However, this specification definitively specifies only wire-level behavior. Actual behavior at a business entity regarding storage and archival of business documents or the management of durable storage resources is implementation dependent and does not affect compliance with this specification.

The only special behavior required from a destination business entity for reliable delivery of BizTalk Documents is to transmit a receipt for each accepted Document in which the <reliability> element of the <delivery> header entry is present. The receipt must be sent to the transport address specified in the <sendReceiptTo> element. The term "accepted Document" in this case means the Document is recognized as being intended for the destination entity, including Documents that are copies or duplicates of previously received Documents (based on the identity). Documents that are received after the time instant specified in <expiresAt> is past are not accepted and must not be acknowledged with a receipt. Documents that are received after the time instant specified in <receiptRequiredBy> is past but before the time instant specified in <expiresAt> is past are accepted and must be acknowledged with a receipt. The structure of receipts and the required correlation between a Document and its receipt has been described above.

A destination business entity may in addition perform idempotent delivery of BizTalk Documents to the target applications at its own end. This may be a part of the business process agreed upon between the parties involved, or an independent configuration parameter at the destination that is either constant or configurable for each business or service relationship. Idempotent delivery implies that a BizTalk Document received at a destination business entity is delivered exactly once to its intended recipient application, even when it is received multiple times due to transport behavior or transmission retries at the source.

The guideline for duplicate removal to achieve idempotent delivery, when required, is to archive in durable store all BizTalk Documents accepted, at least until they expire (that is until the time instant specified in <expiresAt> is past). It is actually sufficient to archive only the identity of the BizTalk Document, given the requirement that the identity must be universally unique. Note that the duplicate removal process needs to be applied only after a Document is accepted.

8.2 Delivery and Processing Deadlines

It is important to understand the distinction between the two deadlines represented by the receiptRequiredBy and expiresAt elements. The content of receiptRequiredBy is the delivery deadline and the content of expiresAt is the processing deadline. The reason for making this distinction is that business actions often require a nontrivial amount of time for processing. If the action associated with a BizTalk Document is expected to take at most 4 hours to carry out, and must be completed within 12 hours of transmission of the Document, then the delivery deadline for this Document must be 8 hours rather than 12 hours. This flexibility is required in designing realistic business processes. The semantics of the two deadlines can be stated as follows:

• The delivery deadline concerns acceptance of the BizTalk Document by the destination business entity. The acceptance of a Document is defined in section 8.1.2. Acknowledgement of acceptance must be received by the source business entity by the delivery deadline.
• The processing deadline is the point in time beyond which the BizTalk Document, if unprocessed, is null and void. The Document must not be delivered to an application for normal processing or acknowledged in any way by the destination business entity after this point in time.

The following points clarify what happens as each deadline expires:

• When the delivery deadline expires:
• The sending BFC server should notify the sending application if a delivery receipt has not been received. This alerts the sending application to the possibility that the receiving application may not have received the Document, or if it did, may not have enough time to process it.
• There is no essential significance for the receiving BFC server. The receiving server continues to accept and acknowledge Documents past this deadline. This means that receipts may be generated and received past this deadline. In fact, the receiver has no absolute need to know the delivery deadline. However, if the receiving server does not know when the delivery receipt is expected, it may choose to delay sending the receipt for internal optimization reasons, causing unnecessary complications at the sender; or, being aware of these consequences, it may give the receipt transmission the highest possible priority reducing its possibilities for internal optimization. The delivery deadline serves as a useful priority hint to the receiver for sending the delivery receipt, which is especially useful in intermittent connection scenarios.
• When the processing deadline expires:
• There is currently no special behavior recommended for the sending BFC server. In a future version of the BizTalk Framework, the notion of a processing acknowledgement may be added. If this concept were added, the sending BFC server should notify the sending application if a processing acknowledgement is not received before the processing deadline expires. This would alert the sending application to the likelihood that the receiving application has not processed the Document (even if a delivery receipt was received).
• The receiving BFC server will reject all Documents that arrive past the processing deadline. They will not be acknowledged with any kind of receipt and will not be delivered to any application for normal processing.