13.9. Filename Substitution (Globbing)

When evaluating the command line, the shell uses metacharacters to abbreviate filenames or pathnames that match a certain set of characters . The filename substitution metacharacters listed in Table 13.11 are expanded into an alphabetically listed set of filenames. The process of expanding the metacharacter into filenames is also called filename substitution, or globbing . If a metacharacter is used and there is no filename that matches it, the shell treats the metacharacter as a literal character.

Table 13.11. Shell Metacharacters and Filename Substitution

Metacharacter	Meaning
*	Matches zero or more characters
?	Matches exactly one character
[abc]	Matches one character in the set a , b , or c
[!abc]	Matches one character not in the set, not a , b , or c
{a,ile,ax}	Matches for a character or set of characters
[a “z]	Matches for one character in the range from a to z
[!a “z]	Matches one character not in the range from a to z
\	Escapes or disables the metacharacter

13.9.1 The Asterisk

The asterisk is a wildcard that matches for zero or more of any characters in a filename.

Example 13.41.

 1   $  ls  *   abc abc1 abc122 abc123 abc2 file1 file1.bak file2 file2.bak none   nonsense nobody nothing nowhere one  2   $  ls  *.bak   file1.bak file2.bak  3   $  echo a*   ab abc1 abc122 abc123 abc2  

EXPLANATION

1. The asterisk expands to all of the files in the present working directory. All of the files are passed as arguments to ls and displayed.

2. All files starting with zero or more characters and ending with .bak are matched and listed.

3. All files starting with a , followed by zero or more characters, are matched and passed as arguments to the echo command.

13.9.2 The Question Mark

The question mark represents a single character in a filename. When a filename contains one or more question marks, the shell performs filename substitution by replacing the question mark with the character it matches in the filename.

Example 13.42.

 1   $  ls   abc  abc122  abc2  file1.bak  file2.bak  nonsense  nothing  one   abc1  abc123  file1  file2  none  noone  nowhere  2   $  ls a?c?   abc1 abc2  3   $  ls ??   ls: ??: No such file or directory  4   $  echo  abc???   abc122 abc123  5   $  echo ??   ??  

EXPLANATION

1. The files in the current directory are listed.

2. Filenames starting with a , followed by a single character, followed by c and a single character, are matched and listed.

3. Filenames containing exactly two characters are listed, if found. Because there are not any two-character files, the question marks are treated as a literal filename. Such a file is not found, and the error message is printed.

4. Filenames starting with abc and followed by exactly three characters are expanded and displayed by the echo command.

5. There are no files in the directory that contain exactly two characters. The shell treats the question mark as a literal question mark if it cannot find a match.

13.9.3 The Square Brackets

The brackets are used to match filenames containing one character in a set or range of characters.

Example 13.43.

 1   $  ls   abc  abc122 abc2 file1.bak file2.bak nonsense nothing   one  abc1 abc123 file1 file2 none noone nowhere  2   $  ls abc[123]   abc1  abc2  3   $  ls abc[13]   abc1  abc2  4   $  ls [az][az][az]   abc one  5   $  ls [!fz]???   abc1  abc2  6   $  ls abc12[23]   abc122 abc123  

EXPLANATION

1. All of the files in the present working directory are listed.

2. All filenames containing four characters are matched and listed if the filename starts with abc , followed by 1 , 2 , or 3 . Only one character from the set in the brackets is matched.

3. All filenames containing four characters are matched and listed, if the filename starts with abc and is followed by a number in the range from 1 to 3 .

4. All filenames containing three characters are matched and listed, if the filename contains exactly three lowercase alphabetic characters.

5. All filenames containing four characters are listed if the first character is not a letter between f and z ([!f “z] , followed by three of any characters, where ? represents a single character.

6. Files are listed if the filenames contain abc12 followed by 2 or 3 .

13.9.4 Brace Expansion

The curly braces match for any of a list of comma-separated strings. Normally the strings are filenames. Any characters prepended to the opening curly brace are called the preamble , and any characters appended to the closing curly brace are called the postamble . Both the preamble and postamble are optional. There can be no unquoted whitespace within the braces.

Example 13.44.

 1   $  ls   a.c b.c abc ab3 ab4 ab5 file1 file2 file3 file4 file5 foo   faa fumble  2   $  ls f{oo,aa,umble}   foo faa fumble  3   $  ls a{.c,c,b[3-5]}   a.c ab3 ab4 ab5  4   $  mkdir /usr/local/src/bash/{old,new,dist,bugs}  5   $  chown root /usr/{ucb/{ex,edit},lib/{ex?.?*,how_ex}}  6   $  echo fo{o, um}*   fo{o, um}*  7   $  echo {mam,pap,ba}a   mama papa baa  8   $  echo post{script,office,ure}   postscript postoffice posture  

EXPLANATION

1. All the files in the current directory are listed.

2. Files starting with f and followed by the strings oo , aa , or umble are listed. Spaces inside the curly braces will cause the error message Missing } .

3. Files starting with a followed by .c , c , or b3 , b4 , or b5 are listed. (The square brackets can be used inside the curly braces.)

4. Four new directories will be made in /usr/local/src/bash : old , new , dist , and bugs .

5. Root ownership will be given to files, ex and edit , in directory /usr/ucb and to files named ex followed by one character, a period, and at least one more character, and a file called how_ex in directory /usr/lib .

6. Brace expansion will not occur if there are any unquoted spaces within the braces.

7. Brace expansion does not necessarily always cause expansion of filenames. In this example the postamble a is added to each of the strings within the curly braces and echoed back after the expansion.

8. The preamble is the string post , followed by a comma-separated list enclosed within braces. Brace expansion is performed and the resulting strings are displayed.

13.9.5 Escaping Metacharacters

To use a metacharacter as a literal character, use the backslash to prevent the metacharacter from being interpreted.

Example 13.45.

 1   $  ls   abc file1 youx  2   $  echo How are you?   How are youx  3   $  echo How are you\?   How are you?  4   $  echo  When does this line \   > ever end\?   When does this line ever end?  

EXPLANATION

1. The files in the present working directory are listed. (Note the file youx .)

2. The shell will perform filename expansion on the ? . Any files in the current directory starting with y-o-u and followed by exactly one character are matched and substituted in the string. The filename youx will be substituted in the string to read How are youx (probably not what you wanted to happen).

3. By preceding the question mark with a backslash, it is escaped, meaning that the shell will not try to interpret it as a wildcard.

4. The newline is escaped by preceding it with a backslash. The secondary prompt is displayed until the string is terminated with a newline. The question mark ( ? ) is escaped to protect it from filename expansion.

13.9.6 Tilde and Hyphen Expansion

The tilde character was adopted by the bash shell (from the C shell) for pathname expansion. The tilde by itself evaluates to the full pathname of the user 's home directory. ^[7] When the tilde is appended with a username, it expands to the full pathname of that user.

^[7] The tilde character will not be expanded if enclosed in either double or single quotes.

When the plus sign follows the tilde, the value of the PWD (present working directory) replaces the tilde. The tilde followed by the hyphen character is replaced with the previous working directory; OLDPWD also refers to the previous working directory.

Example 13.46.

 1   $  echo ~   /home/jody/ellie  2   $  echo ~joe   /home/joe  3   $  echo ~+   /home/jody/ellie/perl  4   $  echo ~   /home/jody/ellie/prac  5   $  echo $OLDPWD   /home/jody/ellie/prac  6   $  cd   /home/jody/ellie/prac  

EXPLANATION

1. The tilde evaluates to the full pathname of the user 's home directory.

2. The tilde preceding the username evaluates to the full pathname of joe 's home directory.

3. The ~+ notation evaluates to the full pathname of the working directory.

4. The ~ “ notation evaluates to the previous working directory.

5. The OLDPWD variable contains the previous working directory.

6. The hyphen refers to the previous working directory; cd to go to the previous working directory and display the directory.

13.9.7 Controlling Wildcards (Globbing)

If the bash noglob variable is set or if the set command is given a “f option, filename substitution, called globbing , is turned off, meaning that all metacharacters represent themselves ; they are not used as wildcards. This can be useful when searching for patterns containing metacharacters in programs like grep , sed , or awk . If globbing is not set, all metacharacters must be escaped with a backslash to turn off wildcard interpretation.

The built-in shopt command ( bash versions 2.x) also supports options for controlling globbing.

Example 13.47.

 1   $  set noglob  or  set -f  2   $  print * ?? [] ~ $LOGNAME   * ?? [] /home/jody/ellie ellie  3   $  unset noglob  or  set +f  4   $  shopt -s dotglob   # Only available in bash versions 2.x  5   $  echo *bash*   .bash_history .bash_logout .bash_profile .bashrc bashnote   bashtest  

EXPLANATION

1. The “f option is given as an argument to the set command. It turns off the special meaning of wildcards used for filename expansion.

2. The filename expansion metacharacters are displayed as themselves without any interpretation. Note that the tilde and the dollar sign are still expanded, because they are not used for filename expansion.

3. If either noglob is unset or the +f option is set, filename metacharacters will be expanded.

4. The shopt built-in allows you to set options for the shell. The dotglob option allows filenames to be matched with globbing metacharacters, even if they start with a dot. Normally the files starting with a dot are invisible and not recognized when performing filename expansion.

5. Because the dotglob option was set in line 4, when the wildcard * is used for filename expansion, the filenames starting with a dot are also expanded if the filename contains the pattern bash .

13.9.8 Extended Filename Globbing ( bash 2.x)

Derived from Korn shell pattern matching, bash 2.x has included this extended functionality, allowing regular expression-type syntax (see Table 13.12). The regular expression operators are not recognized unless the extglob option to the shopt command is turned on:

  shopt -s extglob  

Example 13.48.

 1   $  shopt -s extglob  2   $  ls   abc      abc122    f1    f3     nonsense   nothing   one   abc1     abc2      f2    none   noone      nowhere  3   $ ls  abc?(12)   abc      abc1      abc2  4   $ ls  abc*([1-5])   abc      abc1      abc122     abc2  5   $ ls  abc+([0-5])   abc1     abc122    abc2  6   $ ls  no@(thingne)   none     nothing  7   $ ls  no!(thing)   none     nonsense   noone     nowhere  

Table 13.12. Extended Pattern Matching

Regular Expression	Meaning
abc?(29)1	? matches zero or one occurrences of any pattern in the parentheses. The vertical bar represents an OR condition; for example, either 2 or 9 . Matches abc21 , abc91 , or abc1 .
abc*([0 “9])	* matches zero or more occurrences of any pattern in the parentheses. Matches abc followed by zero or more digits; for example, abc , abc1234 , abc3 , abc2 , and so on.
abc+([0 “9])	+ matches one or more occurrences of any pattern in the parentheses. Matches abc followed by one or more digits; for example, abc3 , abc123 , and so on.
no@(onene)	@ matches exactly one occurrence of any pattern in the parentheses. Matches noone or none .
no!(thingwhere)	! matches all strings except those matched by any of the patterns in the parentheses. Matches no , nobody , or noone , but not nothing or nowhere .

EXPLANATION

1. The shopt built-in is used to set the extglob (extended globbing) option, allowing bash to recognize extended pattern-matching characters.

2. All the files in the present working directory are listed.

3. Matches filenames starting with abc and followed by zero characters or one of either of the patterns in parentheses. Matches abc , abc1 , or abc2 .

4. Matches filenames starting with abc and followed by zero or more numbers between 1 and 5 . Matches abc , abc1 , abc122 , abc123 , and abc2 .

5. Matches filenames starting with abc and followed by one or more numbers between and 5 . Matches abc1 , abc122 , abc123 , and abc2 .

6. Matches filenames starting with no and followed by thing or ne . Matches nothing or none .

7. Matches filenames starting with no and followed by anything except thing . Matches none , nonsense , noone , and nowhere . The ! means NOT.