The Secrets of the Shell

You can use a number of techniques with the bash shell. For example, you can direct the output of one command to a file or even to another command. The shell enables you to set up aliases to define the commands of your choice. You can also move a running program to the background, which saves you the trouble of opening up another virtual terminal or console.

The bash shell is flexible; there are different ways to manage input to bash commands. For example, two different kinds of wildcard characters help you represent more than one file. And Linux allows you to use three different kinds of quote characters to manage the input to a command.

Other secrets allow you to easily move to any home directory, set aliases that can ease administration, and move up and down the Linux directory tree quickly.

Data Streams

Linux includes three data streams: data goes in, data comes out, and errors go out a different direction. These concepts are also known as standard input ( stdin ), standard output ( stdout ), and standard error ( stderr ). Standard input normally comes from a keyboard entry to a command. For example, if you run the ls c* command, c* is standard input to the ls command.

Standard output is the result of a command. For example, the files that you see after typing ls are standard output (stdout), which is normally directed to your monitor.

If there is no standard output, there may be an error message. This is the standard error data stream, which is also normally directed to your monitor.

There are two basic ways to redirect stdin, stdout, and stderr. You can pipe one of these data streams to another command, or you can redirect one of these streams to or from a file.

Redirecting Input and Output

Normally, standard input comes from a keyboard. But if you already have a file full of data, you don t need to type everything again ”you can simply redirect that file of data with the left- facing arrow (<)to your program. For example, the following command directs the database_data file to the database_program :

 # database_program < database_data 

In many cases, you ll want to save standard output in a file. For example, the following command using the right-facing arrow (>) saves standard output from the ls command to the file named filelist :

 # ls > filelist 

This overwrites any data in the file named filelist . Alternatively, a double right-facing arrow (>>) appends data to the end of the file named filelist :

 # ls >> filelist 

You can combine these redirection arrows in the same command. For example, if the database_program generates a lot of output, you can save it for later analysis:

 # database_program < database_data > database_output 

Standard error output can help you diagnose trouble with a program. For example, if you have a program that runs in the middle of the night, you may want to redirect the standard error stream from this program to a file so that you ll have some clues if something goes wrong. For example, the following command redirects errors to a file named errorlog :

 # database_program < database_data 2> errorlog 

Similarly, you can ensure that the previous contents of the errorlog file are not overwritten with the following command:

 # database_program < database_data 2>> errorlog 

Tip  

When you look at standard errors, be careful with the 2 > or the 2 >>. No space is allowed between these characters.

File Descriptors and Data Streams

This sidebar is for the programmers. When a process in the shell works with a file, it sets up a file descriptor. These are program system calls which help manage that process.

There are three standard file descriptors: 0, 1, and 2. File descriptor 0 corresponds to standard input, or the right-facing arrow (>). File descriptor 1 corresponds to standard output, or the left-facing arrow (<). File descriptor 2 corresponds to standard error, which is represented by 2 > in the bash shell.

 

Input and Output Pipes

Just as you can redirect stdin, stdout, and stderr to and from specific files, you can also pipe these data streams to other commands. If you want to review permissions on a large number of files, you might use two different commands:

 # ls -l > tempfiles # more tempfiles 

The first command takes your current file listing and stores the result in a file named tempfiles . The second command allows you to read the tempfiles file, one screen at a time. Because your file list probably changes frequently, you should delete the tempfiles file as it becomes out of date.

But this is also inefficient. You can combine these commands with a pipe ( ), which is the character above the backslash on a U.S. keyboard. For example, the following command does the work of the previous two:

 # ls -l  more 

The pipe ( ) takes the standard output from the ls -l command and sends the results as standard input to the more command. You don t need to create or delete any temporary files.

Running in the Background

Linux is a multitasking system. When you don t have additional terminals or virtual consoles available, you can still run multiple programs from a single command line. For example, some of the steps in compiling a kernel can take nearly an hour . When you run that program in the background, you don t have to wait to run other programs.

There are two ways to make programs run in the background. For example, let s assume you have a script in your current directory named test . This script starts an alarm in an hour. You want to run test , but you want to keep working while you wait. To do this, you can run the following command:

 # ./test & 

The ampersand (&) sends program execution to the background. The program continues to run, and you are returned to the command-line interface.

Note  

The ./programname command is used to run programs in the current directory. It s the easiest way to run local programs if the current directory is not in your PATH .

Alternatively, if you re running a program to calculate the value of pi to an infinite number of digits, such a program might take a while to complete. If you forget to use the ampersand (&), you will need another way to send the program to the background. The Ctrl+ Z command suspends a running program; the bg command then sends the program to the background.

Special Shell Characters

Special shell characters regulate standard output. You may already have some special characters assigned to you in your shell. To check, run the stty -a command, which leads to output similar to Figure 8.7.

Figure 8.7: Special shell characters

The output shows a number of special characters and settings. In the output, the carat ( ^ ) corresponds to the Ctrl key on your keyboard. For example, the intr = ^C setting means that the Ctrl+ c command interrupts a running program. Some of the default special characters are described in

Table 8.3. These are only defaults; you can customize the special characters that you use for different commands.

Tip  

The way you type a shell character varies from what you see in the output from the stty -a command. For example, the eof character appears as ^D (uppercase D); you actually run the Ctrl+ d command (lowercase d) to exit from the terminal.

Table 8.3: Special Shell Characters

Character	Description
^C	Interrupts and stops a running program.
^\	Sends the quit command.
^D	Stops standard input; exits from a console.
^Z	Suspends a currently running program.

There are also a number of settings with and without a hyphen in front. For example, while an igncr setting would ignore a carriage return, the -igncr setting corresponds to "Don t ignore a carriage return." In other words, when you press Enter on your keyboard, the shell gives you a new prompt. The echo setting means that what you type on your keyboard is seen in the terminal.

You can assign different sets of special characters with the stty command. For example, to suspend a program with Ctrl+ x (instead of Ctrl+ z ), run the following command:

 # stty susp ^X 

Warning  

The stty command can be dangerous. For example, if you were to enter the stty -echo command, anything you typed later would not be shown on the screen. You d have to enter the stty echo command to restore your original configuration. Imagine the frustration if a cracker were to enter stty -echo in a login profile!

Tildes and Home Directories

One key character in the bash shell is the tilde ( ~ ). It represents the home directory of the currently logged-on user . On most standard U.S. keyboards, you can find this character on the same key as the back quote ( ` ), just above the Tab key.

You can use the tilde with most bash shell commands. For example, users can navigate to their own home directories with the cd ~ command. Alternatively, a user can list the files in his or her home directory with the ls ~ command. Other examples are shown in Table 8.4.

Table 8.4: Command Examples with the Tilde ( ~ )

Command	Result
cd ~	Navigates to your home directory.
cd ~/.kde	Moves to the .kde subdirectory of your home directory. For example, if your username is mj, this moves you to the /home/mj/.kde directory.
ls ~	Lists the files in your home directory.
tar czvf homebk.tar.gz ~	Backs up the files in your home directory.
~/yourprogram	Runs the program named yourprogram in your home directory.

This also can be useful in your Linux scripts, as the tilde ( ~ ) can help you configure a script to be useful for all users on your Linux server.

Connecting the Dots

The dot is nearly as important of a tool as the slash in the bash shell. While a single dot ( . ) represents the current directory, a double dot ( .. ) can help you navigate to the parent directory.

You can use these dots with many bash commands. For example, the ls . command lists the files in the current directory, and the ls . . command lists the files in the parent directory.

You can even use the dot to run programs in the current directory. For example, if you re in the /etc/rc.d/init.d directory with the service scripts, you may not want to enter the full directory path for every command. For example, you could run the ./iptables status command to check the current situation with your firewall.

Wildcards

There are two other special characters in Linux commands, which are variations on the Microsoft concept of wildcards. The characters are the asterisk ( * ) and the question mark ( ? ). The asterisk represents any number of numbers or letters . Each question mark represents one alphanumeric character. For example, if you were to run the following command, you d get a list of all files that start with the letter a :

 # ls a* 

If you have a file named a , it would be part of this list. In contrast, if you were to run the following command, you d get a list of all files with two alphanumeric characters starting with a :

 # ls a? 

If you have a file named a , it would not be a part of this list. However, the files named ab , ac , and ad would. You can also perform more complex file searches with commands like the following:

 # ls ?at? 

This command returns files with names like cate , kata , and mate . It would not return files with names like Catherine , matador , or cat .

You can even define special characters in more detail with brackets ( [] ). For example, if you want to see all files in your directory between f0801.tif and f0806.tif , you can run either of the following commands:

 # ls f080[1-6].tif # ls f080[123456].tif 

Tip  

In the world of Linux, the techniques associated with using wildcards are also known as globbing.

Slashes in the Shell

There are forward slashes ( / ) and backslashes ( \ ). A single forward slash represents the root directory. Additional forward slashes, such as those in /etc/rc.d/init.d , help you navigate to subdirectories.

The backslash is a special character. For example, if you wanted to look for an asterisk ( * ) in your /etc/shadow file, you might try the following command:

 # grep * /etc/shadow 

Unfortunately, this command looks for the name of every file in the current directory in the /etc/shadow file.

The problem is that the asterisk is a wildcard, which looks for almost everything, depending on the context. That s where the backslash can help. When you put the backslash in front of a special character, it escapes the meaning of that character.

In other words, the following command actually looks for asterisks in the /etc/shadow file:

 # grep \* /etc/shadow 

The backslash is handy for other situations, such as listing two-word directories such as Microsoft s My Documents . For example, if you ve mounted a Microsoft Windows drive from a remote computer on the /mnt/win1 directory, you might try to list the files in the directory with the following command:

 # ls /mnt/win1/My Documents 

This command looks for two separate directories: /mnt/win1/My and Documents . The problem is the space between the two words My and Documents . But when you add a backslash, the shell ignores the space and returns the list of files in the mounted My Documents directory:

 # ls /mnt/win1/My\ Documents 

Quotes

There are three types of quote characters on your keyboard: the single quote ( ˜ ), the double quote ( " ), and the back quote ( ` ). When applied to standard input, they perform different functions.

The difference between these characters is in how they affect variables , such as $NAME , and shell commands, such as date . With any pair of quotes, the shell sends everything inside the quotes to the command. In the following example, the echo command is used. In detail, the difference is as follows :

Single quotes The shell does not process any variables or commands.

Double quotes The shell processes variables, such as $NAME , but does not process any commands.

Back quotes The shell tries to process every word in quotes as a command. If there are variables, they are evaluated first, and then processed as a command. Thus, if $NAME were in back quotes, it is processed, and then the result is evaluated as a command.

You can see how this works in the following examples. Assume NAME=Michael . Remember, date is a command that returns the current date and time. The first command has no quotes. The shell interprets the $NAME variable, but does not run the date command:

 #  echo Welcome $NAME, the date is date  Welcome Michael, the date is date 

The next command encloses the input in single quotes. This prevents the shell from interpreting any variables or commands:

 #  echo