Standard Input and Output

In Chapter 3, you saw that the output of a command can be sent to your screen, stored in a file, or used as the input to another command. Similarly, most commands accept input from your keyboard, from a stored file, or from the output of another command. This flexible approach to input and output is based on the UNIX System concepts of standard input and standard output, or standard I/O.

Figure 4–1 shows a command that uses standard I/O. The command gets its input through the channel labeled “standard input.” That input can come from your keyboard (the default), a file, or a command. Similarly, the command delivers its output through the channel labeled “standard output.” The output might go to your screen (the default), a file, or another command.

Figure 4–1: A model for standard input and output

The command doesn’t need to know which of these sources the input comes from, or where the output goes. It is the shell that sets up these connections, according to the instructions in your command line. It does this through the I/O redirection mechanisms, which include pipes and file redirection.

A typical use of the pipe feature is the following command:

 $ man find lp

This uses a pipe to send the output from the man command to the lp command, in order to print a hard copy of the manual page for find.

An example of file redirection is the following command:

 $ man find > temp

This saves the output from the man command as the file temp.

Table 4–1 lists the symbols used to tell the shell where to get input and where to send output. These are called the shell redirection operators.

Table 4–1: Shell Redirection Operators

Symbol	Example	Function
|	cmd1 | cmd2	Run cmd1 and send output to cmd2
>	cmd > file	Send output of cmd to file
>>	cmd >> file	Append output of cmd to file
<	cmd < file	Take input for cmd from file
/dev/stdin	cmd /dev/stdin	Take input from keyboard
2>	cmd 2> errorfile	Send standard error to errorfile (ksh, bash)
2>&1	cmd > msgs 2>&1	Send both output and standard error to msgs (sh, ksh, and bash)
/dev/tty	(cmd > /dev/tty) >& error	Redirect output to screen, and error to error (csh, tcsh, and bash)
>&	cmd >& msgs	Send both output and errors to msgs (csh, tcsh, and bash)

Using Pipes

The pipe symbol (|) tells the shell to take the standard output of one command and use it as the standard input of another command. Using pipes to join individual commands together in pipelines is an easy way to use a sequence of simple commands to carry out a complex task.

For example, suppose you want to know if the user named sue is logged in. One way to find out would be to use the who command to list all of the users currently logged in, and to look for a line listing “sue” in the output. However, on a large system there could be many users-enough to make it difficult to find a specific name in the list.

A better solution is to use a pipe to redirect the output of who to grep. As explained in Chapter 19, the grep command searches through its input and prints the lines that match a target pattern.

 $ who | grep sue sue     tty1        Mar 29 23:16

In this case, grep searches for the string “sue” in its standard input (which is the output of who) and prints any lines that contain it. The output shows that sue is currently logged in.

Piping the output from one command to another is like using a temporary, intermediate file to hold the output of the first command, and then using it as the input to the second, but the pipe mechanism is easier and more efficient. A typical use of pipes is to send the output of a command to a printer:

 $ pr *.prog | lp

This pipeline uses lp to print a hard copy of all files with names ending in .prog. But it first uses the formatting command pr to attach a simple header to each file.

You can use pipes to create pipelines of several commands. For example, this pipeline of three commands prints the names of files in the current directory that have the string “jmf” somewhere in the listing:

 $ ls -1 | grep jmf | lp

To make a long pipeline more readable, you can type the commands in it on separate lines. If the pipe symbol appears at the end of a line, the shell reads the command on the next line as the next element of the pipeline. The previous example could be entered as

 $ ls -1 | > grep jmf | > 1P

Note that the > at the beginning of the second and third lines in this example is printed by the shell, not entered by the user. The > is not the file redirection operator. It is the shell’s default secondary prompt, PS2. The shell uses the secondary prompt to remind you that your command has not been completed and that the shell is waiting for more input. PS2 and other shell variables are discussed in the section called “Shell Variables” later in this chapter.

Output Redirection to a File

The > redirection operator sends the output of a command to a file. For example,

 $ ls -l > filelist

causes the shell to save the output of ls -l as the file filelist. If a file with that name already exists in the current directory, it is overwritten-its contents are emptied and replaced by the output of the command. If the file filelist does not exist, the shell creates it before running the command.

As you can see, the > file redirection operator is very similar to the | command redirection operator. The only difference is that > is always followed by a filename, while a | is always followed by a command.

The >> operator appends data to a file without overwriting it. That means that if the file already exists, the new data will be added on to the end. In this example,

 $ cat notes.jan >> research

the shell redirects the standard output from cat and appends it to the file named research. The result is that the contents of notes.jan are added to the end of research, without destroying any other information in the file. As before, if the file doesn’t exist, it will be created.

Input Redirection from a File

Just as you can use the greater than (or right arrow) symbol, >, to redirect standard output, you can use the less than (or left arrow) symbol, <, to redirect standard input. The < symbol tells the shell to interpret the filename that follows it as the standard input to a command.

For example, as you saw in Chapter 2, the command mail (or mailx) can be used to send a message that you type on standard input:

 $ mail [e-mail] Subject: Congratulations! I heard that you finished. Nice work! CTRL-D

You can use the < redirection operator to replace the keyboard with a file as the standard input. In this case, the mail command will send the contents of that file, instead of waiting for you to enter a message at the keyboard:

 $ mail [e-mail] < note

The < tells the shell to run mail with the contents of note as its standard input.

You can redirect input and output at the same time. The following example uses the sort command to take the information in source, alphabetize it, and put the output in dest:

 $ sort < source > dest

The order in which you indicate the input and output files doesn’t matter, so the following example has the same result:

 $ sort > dest < source

Many commands provide a way for you to specify an input file directly as a filename argument. For example, cat allows you to name one or more input files as arguments. Thus, the commands

 $ cat < script.pl

and

 $ cat script.pl

both display the same file on your screen. In the first case, the shell connects script.pl to the standard input that cat reads. In the second case, the cat command opens the file script.pl and reads its input from it.

Standard Input from the Keyboard

You can also force commands to accept input from the keyboard. The logical filename /dev/stdin refers to standard input. When used as an argument to a command, it causes the shell to send the input from the keyboard to the command, in place of a file.

For example, the command sort is used to alphabetize the lines in a file. If you give it the argument /dev/stdin, it will sort the lines you type in from the keyboard:

 $ sort /dev/stdin > guestlist Ben Robin Dan Marissa CTRL-D

This will sort the names alphabetically and save them in the file guestlist. It reads everything you type up to the CTRL-D, which indicates the end of input from your keyboard.

You can also mix input from the keyboard with input from a file. Suppose you want to personalize a form letter by combining typed information (such as the recipient’s name) with stored text:

 $ cat /dev/stdin form_letter > output Dear Sue, CTRL-D

This concatenates input from your keyboard (standard input) with the contents of form_letter, and saves the result in output.

Some commands, including cat, interpret a minus sign (−) as a shortcut for /dev/stdin. The previous example could also be written as

 $ cat - form_letter > output

Standard Error

In addition to standard input and standard output, the shells provide a third member of the standard I/O family, standard error. Standard error provides a second logical channel that a program can use to communicate with you, separate from standard output. As the name suggests, standard error is normally used for displaying error messages. For example, cat prints the following message to standard error when you try to read a nonexistent file:

 $ cat snetmail cat: snetmail: No such file or directory

Standard error is also used to display prompts, labels, help messages, and comments. If you try to delete a file for which you don’t have write permission, rm uses standard error to display the following message telling you that the file is protected:

 $ rm save_file rm: remove write-protected regular file 'save_file'?

Normally you don’t see any difference between standard output and standard error, but when you redirect standard output to a file, standard error remains connected to your screen. This makes sense, since even when you send the output of a command to a file, you probably still want to see any error messages that might occur. For example, if you try to cat a file that doesn’t exist into another file, the error message shows up on your screen, and you know the command didn’t work:

 $ cat snetmail > mailcopy cat: snetmail: No such file or directory

Redirecting Standard Error in sh, ksh, and bash

The Bourne-compatible shells (sh, ksh, and bash) allow you to redirect standard error. You use the same > and >> symbols as for redirecting standard output, but you precede them with a 2. The command

 $ cat *.pl 2> errors

will send any error messages to the file errors. Similarly,

 $ grep Robin guestlist > names 2>> errors

sends the output of the command to the file names, and appends any error messages to errors. Note that no space is allowed between the 2 and the >>.

Using the number 2 to redirect standard error may seem rather arbitrary The 2 is an example of a file descriptor, a number that a program uses to refer to a file. The default file descriptors are

File Descriptor	Name	Default Assignment
0	standard input	your keyboard
1	standard output	your screen
2	standard error	your screen

To redirect standard output and standard error to the same place, you can try to use the same filename twice, like this:

 $ grep Robin guestlist > names 2> names

However, this will overwrite the output file names if there is both output and an error message. A better solution is to use the command

 $ grep Robin guestlist > names 2>&1

In this example, the 2>&1 at the end says “send standard error to the same place as standard output.” So both standard output and standard error end up in the file names. Note that the 2>&1 must be written just like that, without any spaces, and it must go at the end of the line, or the shell will not parse it correctly

Redirecting Standard Error in csh and tcsh

The C shell and extended C shell also allow you to redirect standard error, but a little differently than the Bourne-compatible shells. To redirect both standard output and standard error to the same file, use the symbol >& (or >>&, to append to the file), as in the following example:

 % cat *.pl >& perlscripts

This will also work in bash, but not in sh or ksh.

There isn’t really a way to redirect just standard error in the C shells. A work-around is to first redirect the standard output to a file, and then redirect the remaining standard error to a separate file, as shown:

 $ (ls -1 > /dev/tty) >& errorfile

In this example, the logical filename /dev/tty refers to your current terminal (your screen, or the window in which you are running to shell). The command ls -1 > /dev/tty causes the shell to redirect the standard output from ls to the screen. This is put in parentheses to indicate that it should happen first. Once the standard output has been redirected to /dev/tty, the >& errorfile will cause the shell to redirect the standard error to errorfile.