Project54.Understand Bash Internals

Project 54. Understand Bash Internals

"How does Bash's internal processing affect my command lines?"

Learn More

Project 80 teaches you the intricacies of shell quoting.

This project explores the workings of the Bash shell and shows you how to avoid writing command lines that expand and process incorrectly. It highlights shell special characters that must be escaped on the command line and shows the order in which a command line is parsed and executed.

Shell Special Characters

Particular characters have a special meaning to the Bash shell. If you wish to use them literally, they must be escaped (hidden from the shell) to prevent interpretation. We can do this in any of three ways:

• Enclose in double quotes

• Enclose in single quotes

• Precede with a backslash

Let's illustrate this with an example. We'll set up two shell variables and echo Jan owes me $20. We'll use double quotes, which escape all special characters except variable expansion, and then use backslash to escape the dollar symbol that should be taken literally.

$ name="Jan"; amount=20 $ echo "*** $name owes me \$$amount" *** Jan owes me $20

Table 6.1 lists the shell special characters and their meaning.

Table 6.1. Shell Special Characters

Character Sequence	Meaning
#	Introduces a comment
;	Command separator
{...}	Signifies a command block
(...)	Forces execution in a subshell
&& ||	Logical AND/OR placed between commands
~	Expands to current user's home directory
~user	Expands to user's home directory
/	Directory and filename separator
$var	Expands the shell or environment variable var
`...`	Executes a command, writing output back to the command line
$(...)	Bash's preferred syntax for `...`
$((...))	Evaluates an integer arithmetic expression
((...))	Evaluates integer arithmetic in a condition
'	Strong quote to prevent interpretation of shell special characters
"	Weak quote within which $variable is expanded
\	Take the next character literally (cancels any special meaning)
*	Wildcard pattern-matching operator
[...]	Character-set wildcard
?	Single-character wildcard
&	Forces a command to be executed in the background
< >	Redirects stdin/stdout
|	Constructs a pipeline
!	Pipeline logical NOT

Command Search Order

When Bash parses a command line, it interprets the first word as a command. That command may be an external Unix command, an alias, or perhaps both. Bash considers each possibility and picks the first that fits. The order in which Bash considers possible interpretations affects the meaning of the command line. The order is:

• Aliases.

• Bash keywords (such as if, to start a conditional statement).

• Functions.

• Bash built-in commands. Type help to obtain a list of Bash's built-in commands.

• External Unix commands like ls and nano.

Three Bash built-in commands are available to override the natural order in which commands are considered. They are:

• command, which forces a command to be considered as either a built-in or a Unix executable, ignoring aliases and functions.

• builtin, which forces a command to be considered as a built-in only.

• enable, which enables and disables built-in commands.

Command-Line Processing

You'll find it handy to know the order in which Bash evaluates and expands a command line, if only to understand why a particular command line might fail and how to correct it.

Bash first reads a command line and tokenizes it, dividing it into elements like commands, variables, aliases, and strings. Processing then occurs in the following order:

• Expand aliases and then reparse the command line to expand aliases within aliases.

• Expand braces. Braces signify a command block.

• Expand tilde to the full pathname of your home directory and ~user to the full pathname of the user's home directory.

• Expand $variables, both shell and environment variables.

• Perform command substitution, such as $(date), which executes the date command, and then write the output from the command back to the command line.

• Evaluate $((...)) integer arithmetic expressions.

• Split words. After variable and parameter expansion has taken place, the expanded text is split into words.

• Expand pathnames.

Here are some examples that illustrate the significance of the command-line processing order.

Learn More

Refer to Project 16 for more information on Bash's command search order.

Tokenization and alias expansion are repeated until no more aliases are found. Take advantage of this by defining an alias within an alias. In the example that follows, we define the alias lf, which includes in its definition the alias dir.

$ alias lf=dir $ alias dir='ls -F'

Alias lf will be expanded to dir and then alias dir will be expanded to ls -F.

$ lf Desktop/    Library/   Music/      Public/   backup/ Documents/  Movies/    Pictures/   Sites/

Next, we demonstrate that aliases are expanded before tilde by virtue of the fact that the following example works.

$ alias ld="ls ~/Documents" $ ld AppleWorks User Data    Microsoft User Data    iChats Downloads               Sophie.ichat           text

However, tilde is expanded before variables, which is why the next example fails.

$ doc="~/Documents" $ ls $doc ls: ~/Documents: No such file or directory

Tip

Type Escape Control-e to expand a command line in situ.

$ lf<Escape>Control-e $ dir <Escape>Control-e $ ls -F

The next two examples demonstrate that variables are expanded after aliases, and show what tricks will and will not work. (Aliases lf and dir from the example above must be defined.)

$ alias hi='$Hello' $ Hello='echo "Hello there!"' $ hi "Hello there!" $ command='lf' $ $command -bash: lf: command not found

Finally, we take advantage of the fact that variables are expanded before command substitution takes place.

$ command="date" $ echo $($command) Thu Jul 28 11:03:03 BST 2005