Item 30. Layer Functionality

There are two things that experienced developers often see as lacking in XML: strong typing and a processing model. However, it turns out that the omission of both of these is a key component of XML's extensibility. Provided only that well- formedness can be maintained , local processes can much better fit XML to their needs because they aren't forced into a fixed group of types or a fixed processing model. Item 25 addressed the problems with strong typing. Now let's look at the processing non-model.

XML processing is typically implemented as a chain of operations in which the output from one step becomes input to the next step. Each step performs a single task; for example, validation, schema validation, XInclude resolution, normalization, tree building, canonicalization, and so on. The input to each process is one or more complete XML documents; the output from each process is usually a complete XML document. A process doesn't care how an XML document was created or what its history was before the process received it. Most importantly, each process performs exactly one operation. There is no one order in which these operations take place. SAX filters lend themselves particularly well to this pattern, but it can be implemented on top of DOM, streams of Unicode characters , strings, or combinations of several APIs as well. How the XML document is represented is not as important as the fact that it is an XML document.

Many processes produce an XML document as output (e.g., XInclusion, DTD processing to resolve entities and apply default attribute values). These can be used as direct input for other processes. Other processes produce documents or document fragments (e.g., XSLT, XQuery). Others produce a Boolean result that can be used to determine whether or how to continue processing (e.g., DTD validation, schema validation). Finally, some processes produce a non-XML output that possibly terminates the processing, at least insofar as the XML chain is concerned (e.g., conversion of XSL-FO to PDF).

For example, I wrote my last book, Processing XML with Java (Boston, MA: Addison-Wesley, 2002), in pure XML. Its processing chain looked like this.

1. Resolve XInclude elements to produce a single complete book file containing all the chapters and examples.

2. Validate the complete book (optional).

3. Apply an XSLT stylesheet to split the entire book into chapters.

4. Apply an XSLT stylesheet to each of those chapters to produce an XSL-FO file for each chapter.

5. Run FOP across each of those XSL-FO files to produce PDFs for each chapter.

The key to this process is that each step is independent of all the others. Although I run them in a certain order that makes sense for me, you could choose a different order that produces the effects you want. For example, if I perform validation at all, I perform it after the XInclude elements are resolved because the DTD I'm using (DocBook) doesn't permit any XInclude elements. However, if the DTD you're using does allow XInclude elements, you might want to validate before the XInclusion is performed. You might even want to do both, and if your application uses schemas instead of DTDs, you might want to use different DTDs for before and after validation. In fact, you can validate at other steps in the process as well (e.g., each individual chapter).

For another example, you might perform XInclusion after the XSLT processing instead of before it if your XSLT stylesheet generated XInclude elements that needed to be resolved. You could even resolve the XIncludes, apply an XSLT stylesheet that inserted new XIncludes, and then resolve the XIncludes again. Many complicated transformations, such as this one, are most easily implemented as a sequence of transformations in which the output of one process becomes the input to the next.

You can also split the processing chain into a tree, performing multiple operations on a single document to generate multiple outputs. For example, the actual process I used has a few more steps, all of which begin with the complete, validated , XInclude-resolved document and produce a variety of outputs, as shown in Figure 30-1. Although slightly less common, it's also possible for a process such as XSLT or XInclude to combine multiple documents or generate several output documents. However, the input is always well-formed XML, and the output is always well- formed XML.

Figure 30-1. One Possible Processing Sequence for XML

Not all the operations produce XML. For instance, the transformation to HTML produces a normally malformed document, and the PDF files generated by FOP aren't even close to XML. However, these are terminal operations. Additional processing may take place (for instance, the HTML document might be uploaded to a server; the PDF can be printed), but this is completely outside the scope of XML. Indeed, to some extent all this processing is outside the scope of XML. Each member of the XML family of specifications defines either one kind of XML document or a process that can be applied to XML documents. However, the specifications never say anything about the order or sequence in which these processes are applied to these documents. That's always a decision to be made in each local processing environment. There is no one right processing chain for XML. The exact sequence of operations is a local decision. The combination I've outlined here is just one possibility that I use. You're free to use any sequence that helps you get your work done.

One of the key ideas of this layered approach is to disentangle the confusion that sometimes arises about which processes happen when. For instance, one frequent question is whether validation happens before or after XInclusion. The answer is, "It depends." This is a local decision. It can be before, after, or both, depending on local needs. You can pass the validator a document that contains xi:include elements or one in which those elements have been replaced by the content they point to. Either or both are acceptable.

However, you must remember that none of these tasks happen automatically, at least not in a clean chain. Each such task has to be explicitly requested . More importantly, each task does one thing. When you pass a document containing xi:include elements to a validator, the validator does not replace the xi:include elements with the documents they point to. If you want that to happen, you must pass the original document to an XInclude resolver first, and then pass the output of the XInclude process to the validator. The necessary result of DTD validation is defined by the XML specification. It does not allow any room for resolution of XIncludes. Similarly, the necessary result of schema validation is defined by the XML Schema specification. It also does not allow resolution of XIncludes. Either or both can work on a document that was created by resolving the XIncludes in another document, but the validators themselves do not resolve XIncludes. They treat xi:include elements the same as they do any other element.

Similar statements can be made for other possible processes such as transformation with a stylesheet or XQuery search. In all cases, a document is fed into the process, which performs the single function it's designed to perform. You can then operate on the output document with the same or different process, but always one process at a time. As long as the processes don't run in parallel, there's rarely any confusion about what should happen. However, as soon as engines for one process (e.g., XSLT) begin taking upon themselves the need to perform additional simultaneous processes (e.g., XInclusion) the actual and proper output becomes impossible to predict. Different engines produce different outputs from the same input document, and it's impossible to tell which, if any of them, are correct. Interoperability is possible only as long as each tool restricts itself to its own domain. That's why it's important to implement your own processing chain as a sequence of black boxes in which the output of one is connected to the input of the next, rather than as a single black box that does everything at once.