Automating Tasks with Shell Scripts

Customizing Your Shell Environment and Storing Data

Variables are a way of addressing bits of the computer memory so that we can store random pieces of information in it. It would be difficult to do much productive work with a computer if all we could store in any location was one particular, predetermined piece of information. Variables give us the ability to name a region X, store whatever value we want in X, and change the value whenever we want. Variables in the shell are used both to hold data to be used in commands and programs written in the shell and to control the behavior of certain aspects of the shell. You've already been introduced peripherally to this second use by way of the PATH variable, which affects where the shell looks to find executable programs. We'll go into somewhat more detail on this use in the next section and cover the former later in this chapter.

Setting Environment and Shell Variables

Many shells make a distinction between environment variables and shell variables in one way or another. Both are variables that you can set and use in a shell. The difference is that environment variables are inherited by any programs (such as subshells) that are children of (Unixism for "run by") that shell, whereas shell variables are not inherited. This might not seem a useful distinction, but there are significant uses for each type. Noninherited shell variables don't cost memory and startup time for subshells, and can be expected to be empty in any shell until they are used to store something. Inherited environment variables, on the other hand, must be copied into the memory space of child programs, taking room and time, and they can be used to pass information between a parent shell and programs that it executes.

NOTE

It is traditional to use uppercase variable names for environment variables and lowercase variable names for shell variables, although there is no requirement that this tradition must be followed in your own scripts.

Setting Variables in bash

In bash, all variables start out as shell variables, and then are upgraded to environment variable status using the export command. To set a shell variable, the syntax is as follows:

 <shellvariablename>=<value> 

TIP

There are no spaces between the <shellvariablename>, the = character, or the <value>. Spaces that creep unannounced and unwanted into statements that you're typing will be one of the most common problems you encounter when working in bash.

To set a shell variable named x to contain the value 7, the bash shell expression is simply

 x=7 

To make a shell variable into an environmental variable, the syntax is

 export <shellvariablename> 

To make the shell variable x (that we previously set to contain the value 7) into an environment variable, the shell expression is

 export x 

This makes the variable x available to other programs we start from within this shell.

Setting Variables in tcsh

In tcsh one uses an explicit command to tell the shell to put a value into a variable, instead of just an = sign. Setting shell variables uses a different command than setting an environment variable: Shell variables use the command set, and environment variables use the command setenv. To set a shell variable, the syntax is as follows:

 set <shellvariablename> = <value> 

To set a shell variable named x to contain the value 7, the shell expression is simply

 set x=7 

TIP

tcsh is not as particular as bash about spaces, but it does require that the spaces be balanced. That is, you can use either x=7 or x = 7, but not x= 7 or x =7. The error that tcsh reports if you use either of these latter two constructs is a seemingly irrelevant complaint about a variable name needing to start with a letter.

To set an environment variable to a particular value, the syntax is

 setenv <environmentvariablename> <value> 

To set an environment variable named Y to contain the value 8, the shell expression is

 setenv Y 8 

Note that with setenv, no = sign is used.

CAUTION

In tcsh it's quite possible for you to create a shell variable and an environment variable with the same name, each containing a distinct value. In this case, most shell commands will see the variable as having the value of the shell variable rather than the value of the environment variable.

Using Shell and Environment Variables

Both shell and environment variables are addressed for use by the prepending of a $ sign before the variable names. A simple demonstration can be accomplished with the echo command, which prints to the STDOUT of the shell, the value of the expression following it.

NOTE

Observe the shell prompt in the command-line examples in this chapter. It's probably not what you're used to seeing as a shell prompt. The shell prompt can be customized to a significant extent. See Table 15.2 (bash) or Table 15.4 (tcsh) in this chapter, and the man pages for bash and tcsh for information on how you can customize your prompt to include the information you find most useful. For many of the examples in this chapter, I'm using a shell prompt that includes a sequentially increasing number. This number is useful not only for demonstrations such as are being done in this chapter, but also for access to the command history list, discussed later in this chapter.

Table 15.2. tcsh Syntax Options for Setting Shell and Environment Variables

Expression syntax	Effect
<word>	Used as a part of a command-line expression, <word> is a string of nonwhitespace characters, or a quoted string that possibly contains spaces.
$<variable>	Expands to the contents of <variable>. This will typically be a <word> or <wordlist>. $<variablename> preferentially expands to the value of the shell variable by the name <variablename> if both shell and environment variables with this name exist.
set <variable> = <word>	Sets the value of <variable> to <word>.
set <variable>[n] = <word>	Treats <variable> as an array of words, and sets the nth value to <word>. This has the effect of setting the nth word of a wordlist to <word>.
setenv <variable> <word>	Sets the environment variable <variable> to contain the value <word>. Most variable manipulations must be done in shell variables, and the values transferred into environment variables if needed.
@ <variable> = <expression>	Treats <expression> as a mathematical expression, attempts to evaluate it, and assigns the result to <variable>. Most forms of errors in attempts at this result in the response @: Expression Syntax.
@ <variable> = ( <expression> )	Same as the previous item. Parentheses can be used to order the execution of parts of the expression, and it's frequently helpful to use them around any expression on general principles.
@ <variable>[n] = <expression>	Treats <variable> as an array, and sets the nth value of it to the value of <expression>.

Table 15.4. The tcsh Reserved Shell Variables

Shell Variable	Effects
addsuffix	Controls addition of / to the end of directory paths and spaces after normal filenames when expanded by shell filename autocompletion.
afsuser	If set, this is the username to autologout under Kerberos authentication.
ampm	If set, shows time in 12-hour AM/PM format.
argv	The list of arguments passed to the shell on startup.
autocorrect	If set, attempts to fix command misspellings.
autoexpand	If set, passes command completion attempts through the expand-history processor. (See the tcsh man page for more details.)
autolist	If set, lists possible expansions for autocompletion if the expansion is ambiguous. If the value is set to ambiguous, lists possibilities only when an autocompletion attempt does not add any new characters.
autologout	The number of minutes of inactivity before autologout. Optionally, the number of minutes before automatic locking of the terminal.
backslash_quote	If set, backslashes (\characters) are automatically inserted before any backslash or quote character in a command completion.
cdpath	A list of directories in which cd should look for subdirectories if they aren't in the current working directory.
color	If set, enables color display for the ls command and shell built-in command ls-F.
command	If set, contains the command that was passed to the shell with a -c flag.
complete	If set to enhance, completion ignores filename case and considers periods, hyphens, and underscores to be word separators, and hyphens and underscores to be equivalent.
correct	If set to cmd, attempts automatic spelling correction for commands. If set to complete, commands are automatically completed. If set to all, the entire command line is corrected.
cwd	The full path of the current working directory.
dextract	If set, pushd +n extracts the nth subdirectory from the stack, rather than rotating it to the top.
dirsfile	The default location in which dirs -S and dirs -L look for their history.
dirstack	An array of all directories in the directory stack.
dspmbyte	If set to euc, enables display and editing of EUC-Kanji (Japanese) code. If set to sjis, enables display and editing of Shift-JIS (Japanese) code. Other options are available see the tcsh man page for more details.
dunique	If set, pushd removes any instances of the pushed directory from the stack, before pushing it onto the top of the stack.
echo	If set, each command and its arguments are echoed to the terminal before being executed.
echo_style	The style of the echo built-in. May be set to bsd, sysv, both, or none to control the behavior of the echo command. See the tcsh man page for more details on behavior affected.
edit	If set, allows command-line editing.
ellipsis	If set, use an ellipsis to represent portions of the path that won't fit in the prompt.
fignore	List of filename suffixes to be ignored in completion attempts.
filec	An unused tcsh shell variable, included to maintain backward compatibility with csh, which used this variable to control whether completion should be used.
gid	The user owning the shell's real group ID.
group	The user owning the shell's group name.
histchars	A string determining the characters used in history substitution. The first character replaces the default ! character, and the second replaces the default ^ character.
histdup	Controls handling of duplicate entries in the history list. If set to all, only unique history events are entered into the history. If set to prev, a run of identical commands is reduced to a single entry in the history list. If set to erase, a repeat of a command already in the history list removes the previous occurrence from the history.
histfile	The default location in which history -S and history -L look for a history file. If unset, ~/.history is used.
histlit	If set, the shell built-in, editor commands, and history-saving mechanism use the literal (unexpanded) form of lines in the history list.
history	The first word indicates the number of history events to save. The optional second word indicates a format for printing the history. See the tcsh man page for more details on format control strings.
home	Initialized to the home directory of the user. Command-line expansion of ~ refers to this variable for its action.
ignoreeof	If set to the empty string or 0 and the input is a terminal, an end-of-file command sent to the terminal causes the shell to print an error rather than exit.
implicitcd	If set, the shell treats a directory name entered on the command line as though it were entered as the argument of a cd command.
inputmode	Can be set to insert or overwrite to control the behavior of command-line editing.
listflags	Contains command-line flags to include with any used when issuing the ls-F shell built-in.
listjobs	If set, all current jobs are listed when a running job is suspended.
listlinks	If set, the ls-F shell built-in command shows the time of file to which symbolic links point.
listmax	The maximum number of items that the list-choices command-line editor and autocompletion will list without prompting.
loginsh	Set by the shell if it is a login shell.
logout	Set by the shell to normal before a normal logout, automatic before an automatic logout, and hangup if the shell was killed by a hangup signal (typically generated by kill -HUP, or by a terminal connection being interrupted rather than cleanly exited).
mail	The name of the files or directories to check for incoming mail. See both the tcsh and mail man pages for more information on the behaviors controlled by this variable.
matchbeep	Controls whether and when command-line completion rings the bell. Setting it to never prevents all beeps. nomatch beeps when there is no current match. ambiguous beeps when there are multiple matches. notunique beeps when there is an exact match, as well as other longer matches. If unset, the behavior is the same as ambiguous.
nobeep	If set, beeping is completely disabled.
noclobber	If set, the shell attempts to prevent output redirection from overwriting existing files. See the tcsh man page for more details.
noglob	If set, filename substitution and directory substitution are inhibited. Normally used only as a performance enhancement for shell scripts where filenames are already known.
nokanji	If set, disables kanji support so that the meta key is used.
nonomatch	If set, a filename or directory substitution that doesn't match any files does not cause an error.
nostat	A list of directories, or patterns that match directories, that should not be examined for matches during completion attempts.
notify	If set, announces job completions immediately rather than waiting until just before the next command prompt appears.
owd	The previous working directory.
path	A list of directories in which to look for executable commands. The path shell variable is set at startup from the PATH environment variable.
printexitvalue	If set and a program exits with a nonzero status, prints the status.
prompt	The string that is printed as the prompt for command-line input. This can contain both literal strings for display as well as a number of special patterns indicating the substitution of everything from the current directory to the username. See the tcsh man page for the (rather extensive) list of options available.
prompt2	The string to use for the inner prompt in while and foreach loops. The same format sequences as used in the prompt variable may be used in prompt2.
prompt3	The string to use for prompting regarding automatic spelling corrections. The same format sequences as used in the prompt variable may be used in prompt2.
promptchars	If set, specifies a pair of characters to substitute between for a shell prompt when a normal user and when su-ed to the super user.
pushdtohome	If set, pushd without any arguments is equivalent to pushd ~.
pushdsilent	If set, pushd and popd don't print the directory stack.
recexact	If set, completion is finished with an exact match even if a longer one is available.
recognize_only_executables	If set, command listings display only executable files in the path.
rmstar	If set, the user is prompted before rm * is allowed to execute.
rprompt	The string to print on the right side of the screen when the prompt is displayed on the left. This prompt accepts the same formatting controls as the prompt variable. In your author's opinion, this is a bizarre shell capability.
savedires	If set, the shell does a dirs -S before exiting.
savehist	If set, the shell does a history -S before exiting.
sched	The format in which the sched built-in prints scheduled events. The string format is the same as that for prompt.
shell	The file in which the executable shell resides.
shlvl	The nested depth of the current shell beneath the login shell for this session.
status	The status returned by the last command to exit.
symlinks	Can be set to several different values to control the resolution of symbolic links. See the tcsh man page for more details.
tcsh	The version number of the tcsh shell.
term	The terminal type currently being used to work in the shell.
time	If set to a number, executes the time built-in after any command that takes longer than that number of seconds. Can also control the format of the output of the time commands so executed. See the tcsh man page for further information.
tperiod	The period, in minutes, between executions of the tcsh special alias, periodic.
tty	The name of the tty for the current terminal, or empty if the current shell is not attached to a terminal.
uid	The user's real numeric user ID.
user	The user's login name.
verbose	If set, causes the words of each command to be printed after any history substitution. Can be set on startup by executing the shell with the -v command.
version	The shell's version ID stamp, as well as a considerable amount of information regarding compile-time options that were specified when the shell was compiled. See the tcsh man page for more information on interpreting the output.
visiblebell	If set, flashes the screen instead of using an audible terminal bell.
watch	A list of user/terminal pairs to watch for logins and logouts.
who	The format string for watch messages. See the tcsh man page for specific format information.
wordchars	A list of nonalphanumeric characters to be considered part of a word by the command-line editor.

Using Variables in bash

The shell use of variables is simultaneously both rather simple and often annoying in its requirement for attention to detail. Variables are case sensitive, and the spacing between variables and operators such as the = sign in assignment statements is critical to the correct functioning of the statement. The habits necessary to work with shell variables however aren't hard to develop, and it's easy enough to experiment and try again if something doesn't work:

 brezup:ray Documents 351 $ echo "Hi There" Hi There brezup:ray Documents 352 $ echo x  x brezup:ray Documents 353 $ echo $x brezup:ray Documents 354 $ x=7 brezup:ray Documents 355 $ echo x x brezup:ray Documents 356 $ echo $x 7 brezup:ray Documents 357 $ Y=8 brezup:ray Documents 358 $ export Y brezup:ray Documents 359 $ echo $y brezup:ray Documents 360 $ echo $Y 8 brezup:ray Documents 361 $ z=$x+$Y 7+8 brezup:ray Documents 362 $ let z=$x+$Y brezup:ray Documents 363 $ echo $z 15 

Here, a shell variable x and an environment variable Y have been set to values 7 and 8, respectively, and their values have been printed to the terminal. There are a few lines of "mistakes" interspersed to demonstrate the behavior of the shell if you don't get your variable names quite right when you're trying to use them. It's important to note that bash doesn't provide error diagnostics for a simple misuse such as asking for the value of a variable that's never been set. This can make finding errors in scripts more challenging.

Notice the result of the command numbered 361? bash automatically expands variables to their values (that is, replaces them in the expression with whatever value they contain), but it doesn't automatically evaluate arithmetic expressions. Because of this, the assignment done in command 361, z=$x+$Y, is treated as a string assignment: $x and $Y are replaced by their values to the left and right of the + sign, and the resulting string is stored in z. To tell bash to treat the expression as an arithmetic expression instead, the let command is used. This signals bash that the following arguments should be expanded, and then evaluated arithmetically, rather than as a string.

bash also supports the notion of array variables. An array can be created simply by assigning values to subscripted variables:

 brezup:ray Documents 364 $ z[3]=12 brezup:ray Documents 365 $ z[$x]=14 brezup:ray Documents 366 $ echo $z[3] 15[3] 

NOTE

Remember, the value of $x and $z have been set earlier.

Unfortunately, the syntax for retrieving the values isn't quite what you'd probably like it to be. Command number 366 reports 15[3] in the preceding example because $z has previously been set to the value 15, and bash (annoyingly) expands that value, instead of noticing that the [3] calls up an array subscript of z.

 brezup:ray Documents 367 $ echo ${z[3]} 12 brezup:ray Documents 368 $ echo $z 15 brezup:ray Documents 369 $ z[$x+Y]=2 brezup:ray Documents 370 $ echo ${z[15]} 2 

When setting an array value, the variable may be specified as <varname>[<subscript>], but when using it, curly braces must be placed around the <varname>[<subscript>] portion to prevent the shell from interpreting it as $<varname> followed by the string [<subscript>]. Inside the square braces specifying the subscript, expressions are treated as arithmetic and evaluated to determine the final subscript value.

TIP

Peculiarly, the arithmetic expansion inside [] means that you don't need to use a $ in front of a variable name, when you want to use that variable's value inside the [] braces. This allows the expression z[$x+Y]=2 to be written as z[x+Y]=2 and z[$x+$Y]=2 as well. Leaving out the $ is a good way to make what you write visually confusing, so we recommend against this, even though it's valid syntax.

Accessing the zeroth value of an array variable is the same as accessing the variable without an array subscript:

 brezup:ray Documents 371 $ echo ${z[0]} 15 brezup:ray Documents 372 $ z[0]=32 brezup:ray Documents 373 $ echo $z 32 

Finally, you can force the shell to recognize a portion of a command line as some variable and a portion as some other information by insulating the variable name with curly braces. For example, if you have a variable named zz and a variable named zzygy, you'd have trouble printing the value of $zz, followed by the characters ygy without this facility:

 brezup:ray Pictures 453 $ zz="howdy neighbor" brezup:ray Pictures 454 $ zzygy="plonk" brezup:ray Pictures 455 $ echo $zz howdy neighbor brezup:ray Pictures 456 $ echo $zzygy plonk brezup:ray Pictures 457 $ echo ${zz}ygy howdy neighborygy 

The most common ways of using shell and environment variables in bash command lines are shown in Table 15.1.

Table 15.1. Useful bash Syntax Options for Setting and Accessing Shell and Environment Variables

Expression syntax	Effect
<word>	Used as a part of a command-line expression, <word> is a string of nonwhitespace characters or a quoted string that possibly contains spaces.
<variable>	The name of a variable. Variable names are composed of alphanumeric and underscore characters and begin with an alphabetic character or underscore.
$<variable>	Expands to the contents of <variable>. This will typically be a <word> or <wordlist>. A <wordlist> is a series of words separated (in the default case) by spaces.
${<variable>[n]}	Expands the nth value of the array-variable named <variable>. The curly braces are required.
<variable>=<word>	Sets the value of <variable> to <word>.
<variable>[n]=<word>	Treats <variable> as an array of words, and sets the nth value to <word>. This has the effect of setting the nth word of a wordlist to <word>.
<variable>[<expression>]=<word>	Treats <variable> as an array of words and <expression> as an arithmetic expression. Evaluates <expression> and sets the <expression>th value of <variable> to <word>.
export <variable>	Makes <variable> available to other processes as an environment variable.
let <variable>=<expression>	Treats <expression> as a mathematical expression, attempts to evaluate it, and assigns the result to <variable>.
let <variable>=( <expression> )	Same as the previous item. Parentheses can be used to order the execution of parts of the expression, and it's frequently helpful to use them around any expression on general principles.
let <variable>[n]=<expression>	Treats <variable> as an array and sets the nth value of it to the value of <expression>.

CAUTION

The spacing between the parts of a command, like the let z=$x+$Y command in the example, is one of the largest sources of difficulty to the beginning shell programmer. Making matters worse, the spacing that's required is different between shells, and what's required in one, breaks others. In bash, the lack of spaces between "words" that are being operated on for example the z, the =, $x, $y, and the + symbol are critical to the command being understood properly. The only place where there's much leniency is between the let and the arithmetic expression part of the command. Inserting spaces in other places will cause the arithmetic expression to be evaluated differently than you intend or cause the entire command to fail.

To demonstrate the difference between shell and environment variables, you can create a subshell and test the variables you used in the previous example in it:

 brezup:ray Documents 373 $ bash brezup:ray Documents 151 $ echo $x brezup:ray Documents 152 $ echo $Y 8 

As you can see, after the subshell is started (notice that the command number in the prompt drops to 151 the size of the retained history prior to the parent shell), the environment variable Y maintains its value and the shell value x goes back to being undefined.

Using Variables in tcsh

tcsh syntax for everything but the most basic variable operations is similar to bash syntax, but not quite identical.

 brezup Documents 200> echo "Hi There" Hi there brezup Documents 201> echo $x tcsh x: Undefined variable. brezup Documents 202> set x=7 brezup Documents 203> echo $x 7 brezup Documents 204> setenv Y 8 brezup Documents 205> echo $y tcsh y: Undefined variable. brezup Documents 206> echo $Y 8 brezup Documents 207> @ z = ( $x + $Y ) brezup Documents 208> echo $z 15 

Here, a shell variable x and an environment variable Y have been set to values 7 and 8, respectively, and their values have been printed to the terminal. tcsh is more strict about variable name use than bash, and complains when you try to access the value of a variable that hasn't previously been set. The @ command is a tcsh shell built-in command, similar to the set command. However, the set command treats all its arguments as strings, whereas the @ command treats them as numbers, allowing math operations such as +, -, /, and *. The @ command, like the set command, sets a shell variable (or creates it if it does not exist). The set and @ commands can also be used as [set or @] <variablename>[n] = <expression>. In this form, the command attempts to treat the variable <variablename> as an array and set item n (the nth word, if echoed) to the value of <expression>. Table 15.2 lists the most frequently used methods for setting variable values.

CAUTION

The spacing between the parts of a command, like the @ z = ( $x + $Y ) command on line 207 of the preceding example, is one of the largest sources of difficulty to the beginning shell programmer. The spaces between "words" that are being operated on here the $x, the $y, and the + symbol are critical to the command being understood properly. You can have more spaces, but if you remove a space, the words become indistinct, and the shell becomes confused. For example, if you remove the space between the $x and the + sign, tcsh will no longer see a variable named $x, a + sign, and a variable named $y. Instead, it will see a variable named $x+ and a variable named $y, with no mathematic operation between them. This turns out to be two errors because the + symbol isn't a valid part of a variable name, and some math operation is required in the expression.

Peculiarly, while tcsh requires balanced spaces around the = character in set and setenv statements, it does not require balanced spaces around the = in @ statements. There must be a space between the @ and the variable into which the value is being placed, but you can write @ z=3, @ z = 3, @ z= 3, and @ z =3 to the same effect.

To demonstrate the difference between shell and environment variables, you can create a subshell and test the variables you used in the previous example in it:

 localhost ray 210> tcsh /Users/ray localhost ray 151> echo $x x: Undefined variable. localhost ray 152> echo $Y 8 

As you can see, after the subshell has been started (notice that the command number in the prompt drops to 151 the size of my retained command history list), the environment variable Y maintains its value and the shell value x becomes undefined.

Reserved Variables in the Shell

As mentioned earlier, certain shell and environment variables are reserved by the shell and used to either report various values to the user or to control the behavior of some parts of the shell or of programs that run as children of the shell. Tables 15.3 and 15.4 list the bash and tcsh shell variables that affect the behavior of bash, tcsh, and a few intimately related programs. Remember that any program you run in a shell may be additionally affected by environment variables. For example, the man command determines where to look for man pages by examining the MANPATH environment variable. This variable isn't set or controlled by either bash or tcsh, but if you set this environment variable to some path in your shell, the man command will inherit it and search in that path for man pages. Because every program may independently choose to examine any environment variables it chooses, it's best to look at the man pages for any programs to determine whether there are environment variables with which you can affect the program's behavior.

Table 15.3. The bash Reserved Shell Variables

Shell Variable	Effects
BASH	Expands to the full filename used to invoke this instance of bash.
BASH_VERSINFO	A read-only array variable whose members hold version information for this instance of bash. The values assigned to the array members are as follows:
		BASH_VERSINFO[0]	The major version number (the release)
		BASH_VERSINFO[1]	The minor version number (the version)
		BASH_VERSINFO[2]	The patch level
		BASH_VERSINFO[3]	The build version
		BASH_VERSINFO[4]	The release status (for example, beta1)
		BASH_VERSINFO[5]	The value of MACHTYPE
BASH_VERSION	Expands to a string describing the version of this instance of bash.
COMP_CWORD	An index into ${COMP_WORDS} of the word containing the current cursor position. This variable is available only in shell functions invoked by the programmable completion facilities.
COMP_LINE	The current command line. This variable is available only in shell functions and external commands invoked by the programmable completion facilities.
COMP_POINT	The index of the current cursor position relative to the beginning of the current command. If the current cursor position is at the end of the current command, the value of this variable is equal to ${#COMP_LINE}. This variable is available only in shell functions and external commands invoked by the programmable completion facilities.
COMP_WORDS	An array variable consisting of the individual words in the current command line. This variable is available only in shell functions invoked by the programmable completion facilities.
DIRSTACK	An array variable containing the current contents of the directory stack. Directories appear in the stack in the order they are displayed by the dirs builtin. Assigning to members of this array variable may be used to modify directories already in the stack, but the pushd and popd builtins must be used to add and remove directories. Assignment to this variable will not change the current directory. If DIRSTACK is unset, it loses its special properties, even if it is subsequently reset.
EUID	Expands to the effective user ID of the current user, initialized at shell startup. This variable is read-only.
FUNCNAME	The name of any currently executing shell function. This variable exists only when a shell function is executing. Assignments to FUNCNAME have no effect and return an error status. If FUNCNAME is unset, it loses its special properties, even if it is subsequently reset.
GROUPS	An array variable containing the list of groups of which the current user is a member. Assignments to GROUPS have no effect and return an error status. If GROUPS is unset, it loses its special properties, even if it is subsequently reset.
HISTCMD	The history number, or index in the history list, of the current command. If HISTCMD is unset, it loses its special properties, even if it is subsequently reset.
HOSTNAME	Automatically set to the name of the current host.
HOSTTYPE	Automatically set to a string that uniquely describes the type of machine on which bash is executing.
LINENO	Each time this parameter is referenced, the shell substitutes a decimal number representing the current sequential line number (starting with 1) within a script or function. When not in a script or function, the value substituted is not guaranteed to be meaningful. If LINENO is unset, it loses its special properties, even if it is subsequently reset.
MACHTYPE	Automatically set to a string that fully describes the system type on which bash is executing, in the standard GNU cpu-company-system format.
OLDPWD	The previous working directory as set by the cd command.
OPTARG	The value of the last option argument processed by the getopts builtin command.
OPTIND	The index of the next argument to be processed by the getopts builtin command.
OSTYPE	Automatically set to a string that describes the operating system on which bash is executing.
PIPESTATUS	An array variable containing a list of exit status values from the processes in the most recently executed foreground pipeline (which may contain only a single command).
PPID	The process ID of the shell's parent. This variable is read-only.
PWD	The current working directory as set by the cd command.
RANDOM	Each time this parameter is referenced, a random integer between 0 and 32767 is generated. The sequence of random numbers may be initialized by assigning a value to RANDOM. If RANDOM is unset, it loses its special properties, even if it is subsequently reset.
REPLY	Set to the line of input read by the read builtin command when no arguments are supplied.
SECONDS	Each time this parameter is referenced, the number of seconds since shell invocation is returned. If a value is assigned to SECONDS, the value returned upon subsequent references is the number of seconds since the assignment plus the value assigned. If SECONDS is unset, it loses its special properties, even if it is subsequently reset.
SHELLOPTS	A colon-separated list of enabled shell options. Each word in the list is a valid argument for the -o option to the set builtin command. The options contained in/displayed by SHELLOPTS are those reported as being on by the command set -o. If this variable is in the environment when bash starts up, each shell option in the list will be enabled before reading any startup files. This variable is read-only.
SHLVL	Incremented by one each time an instance of bash is started.
UID	Expands to the user ID of the current user, initialized at shell startup. This variable is read-only.
BASH_ENV	If this parameter is set when bash is executing a shell script, its value is interpreted as a filename containing commands to initialize the shell, as in ~/.bashrc. The value of BASH_ENV is subjected to parameter expansion, command substitution, and arithmetic expansion before being interpreted as a filename. PATH is not used to search for the resultant filename.
CDPATH	The search path for the cd command. This is a colon-separated list of directories in which the shell looks for destination directories specified by the cd command. A sample value is ".:~:/usr".
COLUMNS	Used by the select builtin command to determine the terminal width when printing selection lists. Automatically set upon receipt of a SIGWINCH.
COMPREPLY	An array variable from which bash reads the possible completions generated by a shell function invoked by the programmable completion facility.
FCEDIT	The default editor for the fc builtin command.
FIGNORE	A colon-separated list of suffixes to ignore when performing filename completion. A filename whose suffix matches one of the entries in FIGNORE is excluded from the list of matched filenames. A sample value is ".o:~".
GLOBIGNORE	A colon-separated list of patterns defining the set of filenames to be ignored by pathname expansion. If a filename matched by a pathname expansion pattern also matches one of the patterns in GLOBIGNORE, it is removed from the list of matches.
HISTCONTROL	If set to a value of ignorespace, lines which begin with a space character are not entered on the history list. If set to a value of ignoredups, lines matching the last history line are not entered. A value of ignoreboth combines the two options. If unset, or if set to any other value than those documented here, all lines read by the parser are saved on the history list, subject to the value of HISTIGNORE. This variable's function is superseded by HISTIGNORE. The second and subsequent lines of a multi-line compound command are not tested and are added to the history regardless of the value of HISTCONTROL.
HISTFILE	The name of the file in which command history is saved (see HISTORY variable). The default value is ~/.bash_history. If unset, the command history is not saved when an interactive shell exits.
HISTFILESIZE	The maximum number of lines contained in the history file. When this variable is assigned a value, the history file is truncated, if necessary, to contain no more than that number of lines. The default value is 500. The history file is also truncated to this size after writing it when an interactive shell exits.
HISTIGNORE	A colon-separated list of patterns used to decide which command lines should be saved on the history list. Each pattern is anchored at the beginning of the line and must match the complete line. (No implicit * is appended.) Each pattern is tested against the line after the checks specified by HISTCONTROL are applied. In addition to the normal shell pattern matching characters, & matches the previous history line. & may be escaped using a backslash (\); the backslash is removed before attempting a match. The second and subsequent lines of a multiline compound command are not tested, and are added to the history regardless of the value of HISTIGNORE.
HISTSIZE	The number of commands to remember in the command history (see HISTORY variable). The default value is 500.
HOME	The home directory of the current user; the default argument for the cd builtin command. The value of this variable is also used when performing tilde expansion.
HOSTFILE	Contains the name of a file in the same format as /etc/hosts that should be read when the shell needs to complete a hostname. The list of possible hostname completions may be changed while the shell is running; the next time hostname completion is attempted after the value is changed, bash adds the contents of the new file to the existing list. If HOSTFILE is set, but has no value, bash attempts to read /etc/hosts to obtain the list of possible hostname completions. When HOSTFILE is unset, the hostname list is cleared.
IFS	The internal field separator that is used for word splitting after expansion and to split lines into words with the read builtin command. The default value is "<space><tab><newline>".
IGNOREEOF	Controls the action of an interactive shell on receipt of an EOF character as the sole input. If set, the value is the number of consecutive EOF characters that must be typed as the first characters on an input line before bash exits. If the variable exists but does not have a numeric value, or has no value, the default value is 10. If it does not exist, EOF signifies the end of input to the shell.
INPUTRC	The filename for the readline startup file, overriding the default of ~/.inputrc.
LANG	Used to determine the locale category for any category not specifically selected with a variable starting with LC_.
LC_ALL	This variable overrides the value of LANG and any other LC_ variable specifying a locale category.
LC_COLLATE	This variable determines the collation order used when sorting the results of pathname expansion and the behavior of range expressions, equivalence classes, and collating sequences within pathname expansion and pattern matching.
LC_CTYPE	This variable determines the interpretation of characters and the behavior of character classes within pathname expansion and pattern matching.
LC_MESSAGES	This variable determines the locale used to translate doublequoted strings preceded by a $.
LC_NUMERIC	This variable determines the locale category used for number formatting.
LINES	Used by the select built-in command to determine the column length for printing selection lists. Automatically set upon receipt of a SIGWINCH.
MAIL	If this parameter is set to a filename and the MAILPATH variable is not set, bash informs the user of the arrival of mail in the specified file.
MAILCHECK	Specifies how often (in seconds) bash checks for mail. The default is 60 seconds. When it is time to check for mail, the shell does so before displaying the primary prompt. If this variable is unset or set to something other than a positive integer, the shell disables mail checking.
MAILPATH	A colon-separated list of filenames to be checked for mail. The message to be printed when mail arrives in a particular file may be specified by separating the filename from the message with a ?. When used in the text of the message, $_ expands to the name of the current mailfile (for example: MAILPATH='/var/mail/bfox?"You have mail":~/shell-mail?"$_ has mail!"'.) bash supplies a default value for this variable.
OPTERR	If set to the value 1, bash displays error messages generated by the getopts built-in command. OPTERR is initialized to 1 each time the shell is invoked or a shell script is executed.
PATH	The search path for commands. It is a colon-separated list of directories in which the shell looks for commands.
POSIXLY_CORRECT	If this variable is in the environment when bash starts, the shell enters posix mode before reading the startup files, as if the --posix invocation option had been supplied. If it is set while the shell is running, bash enables posix mode, as if the command set -o posix had been executed.
PROMPT_COMMAND	If set, the value is executed as a command prior to issuing each primary prompt.
PS1	The value of this parameter is expanded and used as the primary prompt string. The default value is "\s-\v\$ ". We like to use "\h:\u \W \! \$ " or "\h:\u \W \$ ".
PS2	The value of this parameter is expanded as with PS1 and used as the secondary prompt string. The default is "> ".
PS3	The value of this parameter is used as the prompt for the select command.
PS4	The value of this parameter is expanded as with PS1, and the value is printed before each command bash displays during an execution trace. The first character of PS4 is replicated multiple times, as necessary, to indicate multiple levels of indirection. The default is "+ ".
TIMEFORMAT	The value of this parameter is used as a format string specifying how the timing information for pipelines prefixed with the time reserved word should be displayed. The % character introduces an escape sequence that is expanded to a time value or other information. The escape sequences and their meanings are as follows; the braces denote optional portions.
		%%	A literal %
		%[p][l]R	The elapsed time in seconds
		%[p][l]U	The number of CPU seconds spent in user mode
		%[p][l]S	The number of CPU seconds spent in system mode
		%P	The CPU percentage, computed as (%U + %S) / %R
	The optional p is a digit specifying the precision, the number of digits after a decimal point. A value of 0 forces no decimal point or fraction to be output. At most, three places after the decimal point may be specified; values of p greater than 3 are changed to 3. If p is not specified, the value 3 is used. The optional l specifies a longer format, including minutes, of the form MMmSS.FFs. The value of p determines whether the fraction is included. If this variable is not set, bash acts as if it had the value $'\nreal\t%3lR\nuser\t%3lU\nsys%3lS'. If the value is null, no timing information is displayed. A trailing newline is added when the format string is displayed.
TMOUT	If set to a value greater than zero, TMOUT is treated as the default timeout for the read built-in. The select command terminates if input does not arrive after TMOUT seconds when input is coming from a terminal. In an interactive shell, the value is interpreted as the number of seconds to wait for input after issuing the primary prompt. bash terminates after waiting for that number of seconds if input does not arrive.
auto_resume	This variable controls how the shell interacts with the user and job control. If this variable is set, single word simple commands without redirections are treated as candidates for resumption of an existing stopped job. No ambiguity is allowed; if more than one job begins with the string typed, the job most recently accessed is selected. The name of a stopped job, in this context, is the command line used to start it. If set to the value exact, the string supplied must match the name of a stopped job exactly; if set to substring, the string supplied needs to match a substring of the name of a stopped job. The substring value provides functionality analogous to the %? job identifier. If set to any other value, the supplied string must be a prefix of a stopped job's name; this provides functionality analogous to the % job identifier.
histchars	Two or three characters that control history expansion and tokenization. The first character is the history expansion character, the character that signals the start of a history expansion; normally this is the character !. The second character is the quick substitution character, which is used as shorthand for rerunning the previous command entered, substituting one string for another in the command. The default is ^. The optional third character is the character that indicates that the remainder of the line is a comment when found as the first character of a word, normally #. The history comment character disables history substitution for the remaining words on the line. It does not necessarily cause the shell parser to treat the rest of the line as a comment.

The use of bash and tcsh reserved environment variables is similar in each shell: Values that the shells set may be read and used in commands, and the values that the user can specify may be set to control certain aspects of the operation of the shell. For example, in tcsh, we like to customize our command-line prompt using the prompt variable:

 # pwd /Users/ray/Documents # set prompt="$HOST $cwd:t \! >" brezup Documents 151 > 

In bash, we use the PS1 variable:

 bash-2.05b$ pwd /Users/ray/Pictures bash-2.05b$ PS1='\h:\u \W \! \$ ' brezup:ray Pictures 440 $ 

Some of the options that are controllable in the shells are incredibly specific and provide considerable power only to a subset of users who need their unique functionality. Others, such as the tcsh's visiblebell setting or bash's MAILCHECK variable, allow every user to exert a considerable amount of control and apply significant customizations to their commandline environments.

On the other hand, reserved variables that are automatically set by the shell provide values for you to feed into custom tools and commands that you construct at the command line. The availability of variables such as tcsh's $cwd and bash's $PWD are instrumental in automating repetitive tasks in your environment.

NOTE

The VISUAL and EDITOR environment variables don't specifically affect the shell, but are customarily used to affect the operation of many command-line programs. These are used to suggest appropriate editors to use when some command-line program needs you to edit something, and wants to open an editor in which you can work. The VISUAL variable typically specifies an editor to use when you have screen-formatting capability, and the EDITOR variable is used to specify an editor to use when you are working in a shell that has no formatting capability (think of the difficulty with working in an editor like emacs in the days of paper terminals). Thankfully, this is an almost unheard of situation today, and the VISUAL editor should almost always be available and functional, if you have it set. If you don't have VISUAL or EDITOR set, each application will default to its own preferred editor, which can be confusing.

Alternative Variable Addressing Methods

Both bash and tcsh shell and environment variables can also be addressed in a number of ways other than with the simple $<variablename> method used to return the contents of the variable. These alternative addressing methods can provide a range of information about the variable, allowing you to access everything from its contents to a count of the number of characters that it contains. Table 15.5 provides bash alternatives for accessing other information in the shell or other information regarding the variable, such as the number of words in the variable or whether the variable actually has a value. Table 15.6 provides the analogous information for tcsh.

Table 15.5. Alternative Variable Addressing Methods for bash

Addressing a Variable As	Returns
$name	The value of the variable.
${name}
	If the variable contains multiple words, each is separated by a blank. The braces insulate name from characters following it, causing ${zz}ygy to be distinct from $zzygy.
${name[selector]}	Treats name as an array of words and returns only the selected element from the list of words.
$0	Substitutes the name of the shell or of the file from which command input is being read (used in shell scripts).
$number ${number}	Expands to the numberth argument on the commandline when the shell was invoked.
$*	Expands to the complete list of arguments passed to the shell/shell script, starting with argument 1. If IFS is set, and the expansion occurs within double quotes (that is, "$*"), the arguments are separated by the first character of the value of the IFS variable.
$@	Expands to the complete list of arguments passed to the shell/shell script, starting with argument 1. If the expansion occurs within double quotes, each command-line argument is output as a separate word, separated by spaces.
$#	Expands to the number of parameters supplied on the command line.
$?	Expands to the status of the most recent foreground command/pipeline.
$~	Expands to the current option flags, including both those set at shell invocation and those set by the set built-in command.
$$	Expands to the process ID of the running shell. In a subshell executed as part of a command by use of parenthesis on the command line, it expands to the process ID of the parent shell, not the subshell.
$!	Expands to the process ID of the most recently executed background job started by this shell.
${#name}	Expands to the length, in characters of the value of name.
${#@} ${#*}	Expands to the number of command-line parameters passed to the shell.
${#name[*]} ${#name[@]}	Expands to the number of elements in the array name. Note that this is not necessarily equal to the maximum array subscript, but instead counts only populated array positions.
${name:-word}	Expands to the value of name if name is set; otherwise expands to word.
${name:-$name2}	Expands to the value of name if name is set; otherwise expands to the value of name2.
${name:=word}	Expands to the value of name if name is set; otherwise sets name=word and expands to word.
${name:?word}	Expands to the value of name if name is set; otherwise writes word to STDERR. Causes noninteractive shells to exit if an error is generated.
${name:+word}	Expands to the value of word if name is set; otherwise expands to nothing.
${name:offset:len}	Expands to a substring of the value of name, starting from position offset and extending for len characters. If name is @, expands to len values from the command line parameters, starting with the offsetth parameter.
${name:offset}	Expands to a substring of the value of name, starting from position offset and extending to the end of the value.

Table 15.6. Alternative Variable Addressing Methods for tcsh

Addressing a Variable As	Returns
$name	The value of the variable.
${name}	If the variable contains multiple words, each is separated by a blank. The braces insulate name from characters following it, causing ${zz}ygy to be distinct from $zzygy.
$name[selector] ${name[selector]}	Treats name as an array of words and returns only the selected element from the list of words.
$0	Substitutes the name of the file from which command input is being read (used in shell scripts).
$number ${number}	Equivalent to $argv[number]. Remember that argv is a variable containing an array of command-line arguments passed to the shell.
$*	Equivalent to the $argv array.
$?name ${?name}	Substitutes 1 if variable name is set; 0 if it is not (that is, true or false, depending on whether the variable exists).
$?0	Substitutes 1 if the name of the program running the shell is known. This is specifically applicable to shell scripts and is always 0 for interactive shells.
$#name	Substitutes the number of words in name.
${#name}
$#	Equivalent to $#argv.
$%name		Substitutes the number of characters in name.
${%name}
$?	Expands to the status of the most recent foreground command/pipeline. Equivalent to the $status variable.
$$	Substitutes the process number of the parent shell.
$!	Substitutes the process number of the most recent background process started by the shell.
$<	Substitutes a line from STDIN. This can be used to read input from the keyboard into a shell script.

Variable Substitution Modifiers

Along with the capability to set variables to specific values and to manipulate variable values by the use of external programs, the shell also contains some capability to modify variables internally as well. This capability is mainly targeted to modification of command, filename, and path-like contents in variables. For example, this allows you to parse the extension part of a filename off a file with a name such as myfile.jpg keeping either the extension, jpg, or the main name, myfile.

In tcsh these manipulations are an independent mechanism layered on top of the assorted variable addressing methods and effected by appending to the variable one or more sets of a colon followed by a modifier string.

In bash these manipulations are handled by a set of substitution methods that are part of the variable addressing and expansion syntax. The tcsh way of doing it is much less powerful than the bash way, but it's also far more readable and easier to remember for the sorts of manipulations that you'd do at the command line or in simple shell scripts. This is one of the areas of difference in which tcsh fans typically feel that bash is a highly unfriendly user environment.

Because the tcsh syntax is easier to follow, we'll cover it first this time around. Table 15.7 shows the tcsh variable substitution modifiers and their effects. Table 15.8 shows bash equivalents where available and some of the syntax of bash's more general substitution methods.

Table 15.7. Shell and Environment Variable Substitution :<modifier> Options

Modifier String	Effect
h	Removes a trailing pathname component, leaving the head.
t	Removes all leading path components, leaving only the trailing file component.
r	Removes a filename extension .xxx, leaving the head portion of the filename before this.
e	Removes everything from a filename except for the extension.
u	Changes the case of the first lowercase letter to uppercase.
l	Changes the case of the first uppercase letter to lowercase.
s/l/r/	Substitutes l for r. l can be any simple string, as can r.
g	Applies the next modifier to each word, rather than just to the first occurrence.
a	Applies the next modifier as many times as possible to a single word. Beware of creating modification loops with this option.

Table 15.8. bash Shell and Environment Variable Substitution :<modifier> Options

Modifier String	Effect
${name%/*}	Removes a trailing pathname component from name, leaving the head. This is equivalent to tcsh's <variable>:h.
${name##*/}	Removes all leading path components from name, leaving only the trailing file component. This is equivalent to tcsh's <variable>:t.
${name%.+([!/])}	Removes a filename extension such as .xxx from name, leaving the head portion of the filename that occurs before this. This is equivalent to tcsh's <variable>:r.
${name##*.}	Removes everything from filename name except for the extension. This works unless the filename has no extension, but was specified in a path format in which one or more directories have extensions (periods in their names). In that case, it will return the portion of the path after the rightmost period in the full pathname. This is almost equivalent (with the exception of where it doesn't work) to tcsh's <variable>:e.
${name%pattern}	Expands to the value of name, with the shortest match to pattern removed from the right-hand side. This substituion is how the head-of-a-path operator at the beginning of this table is constructed.
${name[@]%pattern}	Expands to a list of all values of the array name, with the shortest match to pattern removed from the right hand side of each.
${name%%pattern}	Expands to the value of name, with the longest match to pattern removed from the right-hand side.
${name[@]%%pattern}	Expands to a list of all values of the array name, with the longest match to pattern removed from the right-hand side of each.
${name#pattern}	Expands to the value of name, with the shortest match to pattern removed from the left-hand side.
${name[@]#paittern}	Expands to a list of all values of the array name, with the shortest match to pattern removed from the left-hand side of each.
${name##pattern}	Expands to the value of name, with the longest match to pattern removed from the left-hand side.
${name[@]##pattern}	Expands to a list of all values of the array name, with the longest match to pattern removed from the left-hand side of each.
${name/pat/repl}	Expands to the value of name, with the first occurrence of pat replaced by repl. If repl is ommitted, pat is replaced by nothingness, deleting the first occurrence of it from the value.
${name[@]/pat/repl}	Expands to a list of all values of the array name, with the first occurrence of pat replaced by repl in each.
${name//pat/repl}	Expands to the value of name, with all occurrences of pat replaced by repl.
${name[@]//pat/repl}	Expands to a list of all values of the array name, with all occurrences of pat replaced by repl in each.

TIP

If the contents of these tables look intimidating, read ahead to the examples, and then come back here to see more specifically what was done. These are really quite powerful capabilities of the shell, and ones that you won't want to be without. They're also not nearly as confusing to use, as they are to explain!

NOTE

Some bash pattern matching expressions are disabled by default. For example the "remove a trailing file extension" syntax shown in Table 15.8 won't work in bash's default configuration because it uses a repeated character class pattern option. (A complete treatment of regular expressions is beyond the scope of this book the bash man page provides some information, but a much better reference is Mastering Regular Expressions from O'Reilly Publishing.) To enable extended pattern options, you need to tell bash to turn on the extglob shell flag-variable by using the command shopt -s extglob.

There are a number of additional behaviors that can be configured in bash using the shopt command, but these move into the realm of being so bash-specific that they're best left for a book specifically on bash. bash users might wonder why these aren't controlled by variables in the shell, as are many other shell behaviors and capabilities. Your authors wonder this too.

As a simple example in tcsh, if the variable x contains /home/ray/testfile.jpg, we can extract and act upon several different parts of this variable by using the modifiers shown in Table 15.7.

 brezup Documents ray 152> set x=/home/ray/testfile.jpg brezup Documents ray 153> echo $x /home/ray/testfile.jpg brezup Documents ray 154> echo $x:h /home/ray brezup Documents ray 155> echo $x:t testfile.jpg brezup Documents ray 156> echo $x:r /home/ray/testfile brezup Documents ray 157> echo $x:e jpg brezup Documents ray 158> echo $x:u /Home/ray/testfile.jpg brezup Documents ray 159> echo $x:s/test/special/ /home/ray/specialfile.jpg brezup Documents ray 171> set y=( /home/ray/testfile.jpg /home/ray/filetest.jpg ) brezup Documents ray 172> echo $y /home/ray/testfile.jpg /home/ray/filetest.jpg brezup Documents ray 173> echo $y:u /Home/ray/testfile.jpg /home/ray/filetest.jpg brezup Documents ray 174> echo $y:gu /Home/ray/testfile.jpg /Home/ray/filetest.jpg brezup Documents ray 175> echo $y:au /HOME/RAY/TESTFILE.JPG /home/ray/filetest.jpg 

In bash, the analogous commands look like this:

 brezup:ray ray 499 $ x=/home/ray/testfile.jpg brezup:ray ray 500 $ echo $x /home/ray/testfile.jpg brezup:ray ray 501 $ echo ${x%/*} /home/ray brezup:ray ray 502 $ echo ${x##*/} testfile.jpg brezup:ray ray 503 $ echo ${x%.+([!/])} /home/ray/testfile.jpg brezup:ray ray 504 $ shopt -s extglob brezup:ray ray 505 $ echo ${x%.+([!/])} /home/ray/testfile brezup:ray ray 506 $ echo ${x##*.} jpg brezup:ray ray 507 $ echo ${x/test/special} /home/ray/specialfile.jpg brezup:ray ray 508 $ y=([0]="/home/ray/testfile.jpg" [1]="/home/ray/filetest.jpg") brezup:ray ray 509 $ echo ${y/test/special} /home/ray/specialfile.jpg brezup:ray ray 510 $ echo ${y[0]/test/special} /home/ray/specialfile.jpg brezup:ray ray 511 $ echo ${y[@]/test/special} /home/ray/specialfile.jpg /home/ray/filespecial.jpg 

bash doesn't include options analogous to tcsh's capitalization controls as built-in functions. To perform a maniupulation such as changing the case of a value in bash, you would pass the value to an external program such as sed or awk and retrieve the processed value back into a shell variable.

NOTE

The four most important things to remember for working with variables in the shell are how to put values into variables, how to make the variables accessible to other shells and programs, the special treatment necessary to use a variable expression as an arithmetic expression, and how to get values back out of the variables you've set.

In bash, putting values in just uses the = sign, making them available to other programs uses the export command, treating them as math requires the let command, and getting values back out uses $, with a number of optional extra syntax bits on the expression.

In tcsh, these are matched by the set command, the setenv command, the @ expression prefix, and the $ prefix for accessing variables.

Nearly everything you want to do with variables will involve permutations of these.

Command History Substitution

As briefly mentioned earlier, good user shells maintain a history of commands that you have executed at the command line. Although we've only mentioned selecting previous commands out of the history by use of the arrow keys up to this point, both bash and tcsh actually provide a number of options for the use of previous commands from the history in more sophisticated ways. Primary among these is the ability to select among the previous commands and substitute new information for previous information in the commands.

In tcsh, the modification strings for variables detailed earlier can be applied to commands in the history, and some additional history-specific modifiers can be used as well.

In bash, the modification expressions for variables don't work on commands in the history, but instead, a set of history-modification commands almost identical to the tcsh universal (variable and history) modification set is available for use on the command history.

The basic form of history substitution in both shells is simply the exclamation point, which indicates that a history substitution is to take place at that point in the command line. The characters following the exclamation point specify which item from the history is to be used and, optionally, what modifications need to be made to it. Table 15.9 lists the history item specifiers that can follow the exclamation point history substitution indicator.

Table 15.9. History Substitution Options for Both bash and tcsh

Item Following ! Character	Meaning to the History Mechanism
n (n is a number)	Executes the item with that number out of the history list.
-n (n is a number preceded by a minus sign)	Executes the command n items before the current one.
# (the pound sign)	The current command. This allows recursion, so be careful! To indicate a modification of the current event, the # sign indicating the current command can be omitted if a substitution modifier is used also.
!	The previous command (equivalent to -1).
s (s is a character)	Executes the most recent command whose first word begins with s.
?s? (s is a string)	The most recent event that contains the string s.
Quick substitution (Do Not Prepend the ! Character)
^pattern^replacement^	Reissues the most recent command, replacing the first occurrence of pattern in the command with replacement.

For example, a user's command history (which can be listed by use of the history command) is shown in part here:

 brezup:ray Documents 543 $ history | tail -5   539  ls -l   540  cp file1.ps file1.ps.bak   541  cp /usr/test/storage/file1.ps ./   542  lpr file1.ps   543  history | tail -5 

We could execute another lpr file1.ps simply by typing !l on a command line. Alternatively, !?ora? would execute the copy from /usr/test/storage by matching the string ora from storage. !! would re-execute the most recent command, which is history at command number 544, but will be whatever I issue as command 544 when I'm at the 545 prompt. !-4 would execute the command 4 prior to the current command, which is currently the copy to file1.ps.bak, but this also changes as more commands are issued. !540 reissues the cp to file1.ps.bak, but this resolution doesn't change over time, and !540 will always produce that result in this instance of the shell. These are shown here:

 brezup:ray Documents 544 $ !l lpr file1.ps brezup:ray Documents 545 $ !?ora? cp /usr/test/storage/file1.ps ./ brezup:ray Documents 546 $ !! cp /usr/test/storage/file1.ps ./ brezup:ray Documents 547 $ !-4 history | tail -5   543  history | tail -5   544  lpr file1.ps   545  cp /usr/test/storage/file1.ps ./   546  cp /usr/test/storage/file1.ps ./   547  history | tail -5 brezup:ray Documents 548 $ !540 cp file1.ps file1.ps.bak 

These commands can be combined with substitution modifiers as detailed earlier to further reduce the amount of typing effort needed. In the history, bash doesn't use its variable substitution syntax, and instead uses the same :<modifier> syntax that tcsh does. See the tcsh variable substitution modifiers in Table 15.7 for history substitution modifiers that work in bash:

 brezup:ray Documents 549 $ !?ora?:s/1/2/ cp /usr/test/storage/file2.ps ./ brezup:ray Documents 550 $ !540:gs/1/2 cp file2.ps file2.ps.bak brezup:ray Documents 551 $ !540:r.newbak cp file1.ps file1.ps.newbak 

Table 15.10 shows some history-specific :<modifier> strings that can be applied to history substitutions.

Table 15.10. History-Specific :<modifier> Options Available in both bash and tcsh

Modifier String	Action
&	Repeat the previous substitution in this position.
p	Print out a history substitution with expanded substitutions, rather than execute the command.
q	Quote the value after this modification, preventing further modifications.
0	The leftmost argument of the command (typically, the command itself).
n	The nth argument of the command.
^	The first argument, equivalent to 1. The colon can be omitted from before this modifier.
$	The last argument. The colon can be omitted from before this modifier.
%	The word matched by an ?s? search. The colon can be omitted from before this modifier.
x-y	A range of arguments from the xth to the yth.
-y	Equivalent to 0-y. In tcsh the colon can be omitted from before this modifier.
*	Equivalent to ^-$, but returns nothing if the command is the only argument. The colon can be omitted from before this modifier.
x*	Equivalent to x-$.
x-	Equivalent to x*, but omits the last word $.

NOTE

These tables and examples cover only the most commonly used history and variable modification options. The bash and tcsh man pages alone would occupy more than 200 pages of this book, and they are tersely written, to say the least. Entire books have been written on effectively using shells, and if you're interested in making the absolute best use of a shell, we recommend that you pick up one book, or even a handful.

Don't let the volume of options available overwhelm you, though. Most people who use Unix don't make use of even 10% of the options shown in the abbreviated discussion here, and are perfectly happy with their productivity at that level. Be aware that these options exist; they can make your life much easier if you find that you need them, but don't feel obliged to try to actually learn them until you do find a need.

Aliases and Shell Functions

The alias command is a simple tool that can help you customize your environment. At its simplest, it is the textual equivalent of the graphical Mac OS icon aliases (or Windows shortcuts) that you're probably already familiar with. alias makes it possible for you specify a new name by which you can refer to an existing command. If you don't like typing history to list your command history, you can use the alias command to make typing h equivalent to typing history.

We could have introduced this command much earlier in the discussion, but the information you have just learned about history and variable substitution makes the alias command much more powerful than just creating alternate names for commands alias can create alternative names, with alternative options for commands, and simultaneously make preprogrammed modifications to the command and its arguments before invoking the target command itself

At its simplest, the alias command has an almost trivial syntax: In bash it's alias <newname>=<definition>, and in tcsh it's alias <newname> <definition>. It accepts no command-line options and has no arguments or flags to control it. To alias h so that it calls history as described earlier, simply use alias h='history' (or in tcsh, alias h 'history').

TIP

Single quotes aren't absolutely necessary here, but they are generally used around the thing to which you want to make an alias to prevent variable and history expansion in the alias. You generally want an alias to use variables as they'd expand at the current command-line prompt that is, when you issue the aliased command, rather than have those variables and substitutions occur when you type the alias command itself.

The real power of being able to compress a collection of typing down into a single manageable command, however, is in what you can do to customize and automate your shell by making use of variable substitutions in those commands. In tcsh this ability is built into the alias command itself, whereas bash makes it available through a separate ability to define functions that behave like (very powerful, but strangely limited in their ability to deal with the command history) tcsh aliases. bash functions are defined much like aliases, but have the syntax <funcname>() { <commands>; } instead of the simpler alias syntax.

For example, there are a number of machines in another domain that I access on a regular basis. It's inconvenient to type slogin oak.cis.ohio-state.edu, slogin shoe.cis.ohio-state.edu, and so on whenever I need to access one of these machines. Using the trivial application of alias, I could change slogin so that I could type something shorter, such as scis instead of slogin. That's a minor improvement, but really not all that much help. This would still leave me typing scis oak.cis.ohio-state.edu, and so on. Using the power of substitution, however, this can be made much more useful.

In bash I can use the shell's capability to parse out command-line parameters into variables named $<num> to get at values that I put on the command line after calling a function:

 brezup:ray ray 506 $ scis() { slogin $1.cis.ohio-state.edu; } brezup:ray ray 507 $ scis oak [e-mail]'s password: brezup:ray ray 507 $ scis pear [e-mail]'s password: 

The first line of this example defines a function that causes bash to execute a little program for me whenever I type scis. The program's extremely short, and consists of only the following: parsing off the first argument after the scis command; building a new expression that consists of slogin, followed by the value of the first argument, with .cis.ohio-state.edu appended to the end; and finally executing the command it has built. The result is that when I type scis <machinename>, the actual command that the shell executes, after going through all the substitutions, is slogin <machinename>.cis.ohio-state.edu.

In tcsh I can use similar facilities to get to command-line arguments, or I can alternatively access the command history. This can get a bit more convoluted visually (in terms of how I construct the command), but allows you more flexibility in generating useful aliases on the fly as you're working in the shell. bash's function paradigm is quite a bit more powerful, but requires significantly more forethought to use. For example, in tcsh, I can use the * modifier to the history (which returns all the arguments from the specified command in the history), executed against the current command in the history (#). Using these, I can pass the arguments given to my alias to another command of my choice. Specifically, I could use an alias such as alias scis 'slogin \!#:*.cis.ohio-state.edu'. That might look a little ugly the first time you see something like it, but it all breaks down into understandable parts with just a little thought.

tcsh supports using the command-line parameters as bash does, but this example should give you an idea of how the command history might be used in constructing useful aliases. In this case, it's using the !# history expansion, which expands to the entire current command line as typed so far, and the :* modifier that pares off the command-word portion and returns the rest of the arguments. The backslash before the history expansion prevents it from being expanded immediately at the prompt when I entered the alias command. Now, all I have to do is type scis oak, and the alias command expands the command to slogin !#:*.cis.ohio-state.edu. The history substitution then replaces the !#:* with the argument given to the command, which in this case is oak. The final command executed is slogin oak.cis.ohio-state.edu.

NOTE

Several comments in the tables on history substitution and modifiers relate to the !#:* expression suggested here, and can make this less ugly looking. Most notably, if a modifier is acting on the current history event, the # can be omitted, and the : can be omitted before *. This history substitution specifier therefore abbreviates to !*, and the alias could therefore be written alias scis 'slogin \!*.cis.ohio-state.edu'.

Also, the !* substitution specifier substitutes the entire remainder of the command line from the current command. This is fine as long as I type scis oak, but if I type scis oak apple pear, the expansion is probably not going to be what I want. I could limit it to the first argument to the command with the ^ modifier, or the last argument with the $ modifier, instead of the * all-arguments modifier I have used. It's a little bit sloppy, but I normally find it sufficient to just use the all modifier and to remember to issue commands within the restrictions that doing so imposes. For me, this is easier than remembering to use the proper modifier when I need it, but you're welcome to use whatever you find easiest.

NOTE

Nothing in the bash documentation suggests that history substitution isn't possible inside functions, but there doesn't seem to be a way to make it work. If you try to build a history- substituting function in bash, you'll find that it either substitutes the history at the moment of the function creation rather than at the point when you invoke it, or that it escapes the special characters that should cause history substitution and just prints them out instead of expanding them to the proper substitution. Maybe it's a bug, or maybe we just can't figure out the proper syntax.

Because bash undergoes frequent revisions, it's not unusual for portions of the behavior to change between releases. By the time you're reading this, bash might support history substitution inside functions as well.

To remove an alias, simply use the unalias command on the <newname> that you've created for your command. bash doesn't seem to have the facility to undefine functions that you've created.