functions of Program 6.3.

6.2 Pipelines

Section 4.7 explains how a process can redirect standard input or output to a file. Redirection allows programs that are written as filters to be used very generally . This section describes how to use redirection with pipes to connect processes together. (You may want to review Section 4.7, which explains how a process can redirect standard input or output to a file.)

Example 6.4

The following commands use the sort filter in conjunction with ls to output a directory listing sorted by size .

 ls -l > my.file sort -n +4 < my.file 

The first option to sort gives the type of sort ( n means numeric). The second option instructs the program to find the sort key by skipping four fields.

The first command of Example 6.4 causes the process that runs the ls -l to redirect its standard output to the disk file my.file . Upon completion, my.file contains the unsorted directory listing. At this point, the second command creates a process to run the sort with its standard input redirected from my.file . Since sort is a filter, the sorted listing appears on standard output. Unfortunately, when the pair of commands completes, my.file remains on disk until explicitly deleted.

An alternative approach for outputting a sorted directory listing is to use an interprocess communication (IPC) mechanism such as a pipe to send information directly from the ls process to the sort process.

Example 6.5

The following alternative to the commands of Example 6.4 produces a sorted directory listing without creating the intermediate file my.file .

 ls -l  sort -n +4 

The vertical bar ( ) of Example 6.5 represents a pipe. A programmer can build complicated transformations from simple filters by feeding the standard output of one filter into the standard input of the other filter through an intermediate pipe. The pipe acts as a buffer between the processes, allowing them to read and write at different speeds. The blocking nature of read and write effectively synchronize the processes.

The connection between ls and sort in Example 6.5 differs from redirection because no permanent file is created. The standard output of ls is "connected" to the standard input of sort through the intermediate communication buffer. Figure 6.1 shows a schematic of the connection and the corresponding file descriptor tables after the processes representing ls and sort establish the connection. The ls process redirects its standard output to the write descriptor of the pipe, and sort redirects its standard input to the read descriptor of the pipe. The sort process reads the data that ls writes on a first-in-first-out basis. The sort process does not have to consume data at the same rate as ls writes it to the pipe.

Figure 6.1. Status of the file descriptor table during execution of Example 6.5.

Program 6.3 shows a program that implements the equivalent of Example 6.5. Figures 6.2 to 6.4 depict the state of the file descriptor table for Program 6.3. In Figure 6.2, the child process inherits a copy of the file descriptor table of the parent. Both processes have read and write descriptors for the pipe. Figure 6.3 shows the file descriptor table after the child redirects its standard output and the parent redirects its standard input, but before either process closes unneeded file descriptors. Figure 6.4 shows the configuration after each process completes the close calls. This is the configuration inherited by execl .

Figure 6.2. Status of the file descriptor table after the fork in Program 6.3.

Figure 6.3. Status of the file descriptor table after both dup2 functions of Program 6.3.

Figure 6.4. Status of the file descriptor table after all close calls of Program 6.3.

Exercise 6.6

Explain why the only return values in Program 6.3 indicate error conditions. Under what circumstances does this program execute successfully?

Answer:

The program executes successfully when both parent and child successfully run execl on their respective programs and these programs complete successfully. If execution reaches one of the return statements of Program 6.3, at least one of the execl calls failed. Once an execl call completes successfully, the program on which execl was run is responsible for the error handling.

Program 6.3 simpleredirect.c

A program to execute the equivalent of ls -l sort -n +4 .

 #include <errno.h> #include <stdio.h> #include <unistd.h> #include <sys/types.h> int main(void) {    pid_t childpid;    int fd[2];    if ((pipe(fd) == -1)  ((childpid = fork()) == -1)) {       perror("Failed to setup pipeline");       return 1;    }    if (childpid == 0) {                                  /* ls is the child */       if (dup2(fd[1], STDOUT_FILENO) == -1)          perror("Failed to redirect stdout of ls");       else if ((close(fd[0]) == -1)  (close(fd[1]) == -1))          perror("Failed to close extra pipe descriptors on ls");       else {          execl("/bin/ls", "ls", "-l", NULL);          perror("Failed to exec ls");       }       return 1;    }    if (dup2(fd[0], STDIN_FILENO) == -1)               /* sort is the parent */        perror("Failed to redirect stdin of sort");    else if ((close(fd[0]) == -1)  (close(fd[1]) == -1))        perror("Failed to close extra pipe file descriptors on sort");    else {       execl("/bin/sort", "sort", "-n", "+4", NULL);       perror("Failed to exec sort");    }    return 1; } 

Exercise 6.7

What output would be generated if the file descriptors fd[0] and fd[1] were not closed before the calls to execl ?

Answer:

No output would be generated. The sort process reads from standard input until an end-of-file occurs. Since it is reading from a pipe, sort detects an end-of-file ( read returns 0) only when the pipe is empty and no processes have the pipe open for writing. As illustrated in Figure 6.4, only the ls program (the child) can write to the pipe. Eventually, this program terminates, and sort (the parent) detects end-of-file. If Program 6.3 omits the close calls, the situation looks like Figure 6.3. When the child terminates, the parent still has file descriptor [4] open for writing to the pipe. The parent blocks indefinitely, waiting for more data.