Pipe and Filter Revisited

Each of the utilities and techniques developed (or discussed, as in the case of XSLT) in the preceding chapters performed a single conversion or transformation. This may be fine for some types of problems, but most of us dealing with legacy systems will probably need more than one transformation. Let's revisit our modified pipe and filter model from Chapter 1. The figure is repeated here as Figure 11.1.

Figure 11.1. Modified Pipe and Filter Style

Each of the things we've built or discussed so far is a filter. All of our legacy format to XML conversion utilities started with initial inputs of legacy files, used parameters from file description documents (or command line options for the basic CSV to XML converter), and produced one or more XML documents as output. The reverse was true for our XML to legacy format converters. XSLT follows this same model by taking a source tree as input, getting parameters from the stylesheet, and producing a result tree as the output. We now need to string these pieces together into systems that solve common conversion problems. In this implementation our conversion utilities and an XSLT processor are the filters, and the file system is the pipe. We'll talk later in the chapter about using a more efficient pipe.

Before we go further, though, some of you might be thinking, "Hey, this is just slow, inefficient, old-fashioned batch processing!" You're exactly right. The degenerate case of the pipe and filter style is indeed batch. But to that I say, "So what?" We're talking about legacy systems here, and most of the import and export operations we're dealing with happen in batch mode.

Sure, a lot of applications really benefit from more sophisticated real-time processing and more tightly integrated interfaces. However, it doesn't make sense to spend a lot of money sending out invoices in real time if you're giving your customers thirty days to pay. This is true not only of our legacy to and from XML conversions and XSLT transformations but also of several other technical solutions to common business problems. We must always consider the overall context in which a system will be used and the overriding consideration of a reasonable return on investment. From this perspective, sophisticated real-time interfaces are often not justifiable and batch is often good enough.

We're coding these little systems in batch mode and we're going to be very happy about it. We'll talk about some more efficient options later in the chapter.