The Pipe-and-Filter Style

4.1.1 Overview

The pattern of interaction in the pipe-and-filter style is characterized by successive transformations of streams of data. Data arrives at a filter, is transformed, and is passed through pipes to the next filter. A single filter can pass data through multiple ports. Examples of such systems are signal-processing systems and systems built using UNIX pipes.

4.1.2 Elements, Relations, and Properties

The pipe-and-filter style, summarized in Table 4.1, provides a single type of componentthe filterand a single type of connectorthe pipe. A filter transforms data that it receives through one or more pipes and transmits the result through one or more pipes. A pipe is a connector that conveys streams of data from the output port of one filter to the input port of another filter. Pipes act as unidirectional conduits, providing an order-preserving, buffered communication channel to transmit data generated by filters. Pipes change none of the data content. In the pure pipe-and-filter style, filters interact only through pipes.

Table 4.1. Summary of the pipe-and-filter style

Elements	Component types: filter. Filter ports must be either input or output ports. Connector types: pipe. Pipes have data-in and data-out roles.
Relations	Attachment relation associates filter output ports with data-in roles of a pipe and filter input ports with data-out roles of pipes and determine the graph of interacting filters.
Computational model	Filters are data transformers that read streams of data from their input ports and write streams of data to their output ports. Pipes convey streams of data from one filter to another.
Properties	Same as defined by the C&C viewtype.
Topology	Pipes connect filter output ports to filter input ports. Specializations of the style may restrict the association of components to an acyclic graph or a linear sequence.

Because pipes buffer data during communication, filters can act asynchronously, concurrently, or independently. Moreover, a filter need not know the identity of its upstream or downstream filters. For this reason, pipe-and-filter systems have the nice property that the overall computation can be treated as the functional composition of the compositions of the filters.

Constraints on composition of elements in this style dictate that pipes must connect output ports to input ports. Specializations of the pipe-and-filter style may also impose other constraints, such as that the architectural graph be acyclic or that the configuration define a linear sequence, or a pipeline.

ADVICE

Useful properties to document for pipes include whether pipes are buffered, how they treat end-of-data, and their blocking behavior when writing to buffered pipes that are filled up or reading from pipes that are empty. Properties of filters can include whether or not each filter is a separate process, and the stream transformation each achieves.

4.1.3 What the Pipe-and-Filter Style Is For and What It's Not For

Systems with a pipe-and-filter style are heavily oriented toward data transformation. Often, pipes and filters constitute the front end of signal-processing applications. These systems typically receive a large amount of data at the initial filter from sensors; the filter compresses the data and performs initial filtering. The next filters reduce the data further and do synthesis across different sensors. The final filter typically allows the application of the data, such as providing input to modeling tools, providing input to visualization tools for user exploration, or providing input to further application. Analyses that are associated with pipe-and-filter systems include deriving the aggregate transformation provided by a graph of filters and reasoning about system performance: input/output stream latency, pipe buffer requirements, and schedulability.

4.1.4 Relation to Other Styles

A C&C view in a pipe-and-filter style is not the same as a data flow projection or a data flow view. In the pipe-and-filter style, "lines" between components represent connectors, which have a specific computational meaning: They transmit streams of data from one filter to another. In data flow projections, the lines represent relations indicating the communication of data between components. The latter have little computational meaning: They simply mean that data flows from one element to the next. This flow might be realized by a connector, such as a procedure call, the routing of an event between a publisher and a subscriber, or data transmitted via a pipe. The reason that these views might be confused is that the data flow projection of a pipe-and-filter style looks almost identical to the original view.

4.1.5 Examples of the Pipe-and-Filter Style

ECS

Figure 4.1, taken from Appendix A, shows a pipe-and-filter primary presentation for the ECS system.

Figure 4.1. ECS viewed as a nonsequential pipe-and-filter system. Data enters from satellites on the left, is processed and refined, and is made available for distribution to the science community on the right. The steps are explained in the view's supporting documentation, which is given in Appendix A but not repeated here.

String-Processing Application

Figure 4.2 shows a simple string-processing application in a pipe-and-filter style. The system is described hierarchically: The filter MergeAndSort is defined by a refinement that is itself a pipe-and-filter system. Properties of the components and connectors are not shown but would list, for example, performance characteristics used by a tool to calculate overall system throughput.

Figure 4.2. A system in the pipe-and-filter style. The refinement of MergeAndSort is itself in this style.