Project4.Learn Bash Shell Basics | Mac OS X Unix 101 Byte-Sized Projects

Project 4. Learn Bash Shell Basics

This project provides an overview of the Bash shell and its features, and introduces techniques for customizing it to enhance your command-line experience. Awareness of the Bash basics will increase your productivity and improve life at the command line.

You'll spend much of your Unix time talking to Bash through Terminal. Both can be customized, and Bash has many features and facilities to help increase your productivity and make your command-line experience more pleasant. This is a large and important subject areaso much so that an entire chapter, Chapter 6, is devoted to it. This project covers enough Bash for everyday use and for tackling the projects in this book. Follow the links given to projects in Chapter 6 when you need more detailed knowledge of a particular subject area.

Entering Commands

Entering a command is the most basic and frequent task you'll perform in Bash, so it makes sense to simplify it as much as possible. To this end, Bash provides auto completion, command-line history, and command-line editing. This section will get you started on all three.

Auto Completion

Auto completion will probably save you more time and typing than any other feature. You need type only the first few characters of a command or filename, enough to make it unambiguous, and Bash will complete the rest for you. For this exercise, change into your home directory. We are going to list Documents, using autocompletion to do most of the typing for us. Type ls and just Do for documents, and then press the Tab key. Bash will complete the directory name for you, and you need only press Return. Seven keystrokes versus thirteen is a big savings.

$ cd $ ls Do<tab> $ ls Documents/

Learn More

Autocompletion is case sensitive, and typing ls do<tab> will not work as you might expect. Project 45 shows you how to make autocompletion case insensitive.

If the filename you type is ambiguousD, for example, could be Desktop or Documentspressing Tab will have no effect. Continue typing until the filename is unambiguous; then press Tab. If you need a hint, press Tab twice, and Bash will list the possible completions for you.

$ ls D<tab><tab> Desktop   Documents $ ls Do<tab> $ ls Documents/

Command names can also be autocompleted by pressing Tab, with the same rules applying for ambiguity and case sensitivity. Thus, the command apropos can be abbreviated apr<tab>.

Command-Line History

Bash remembers every command you type in its command-line history. Press the up arrow key (cursor-up) to recall previous command lines and the down arrow key (cursor-down) to move in the opposite direction. When you exit a shell, all the commands you typed are saved to a file in your home directory called .bash_history, and when you return, they are read back in again. You can view the history by typing

$ history     1 ls -R     2 cd     3 ls     4 ls Documents/     5 history $

Previous command lines can be recalled and edited. Press Control-r; then start typing a command. As you type, the history will be reverse-searched for the first line that matches what you have typed so far. For example, press Control-r and then type ls. Assuming that you issued command ls Documents recently, Bash will complete the command line for you. Press Return to execute it.

$ Control-r (reverse-i-search)`ls': ls Documents/

If this is not the command you are looking for, either type more characters to identify the command fully or press Control-r repeatedly until the desired command is found. Each press of Control-r causes Bash to search farther back in history for a matching command.

Tip

It's not necessary to type the first letters of the command you wish to search for. Pressing Control-r and typing Doc will initiate a search for previous commands that contain Doc, not just those that begin with Doc.

Command-Line Editing

Any command recalled from the history can be edited. After pressing Control-r to recall a command, press Escape to edit it instead of Return. The left and right arrow keys, and the Backspace keys, work as expected. Useful editing keystrokes include

Control-a to move to the start of the line
Control-e to move to the end of the line
Control-k to delete from the cursor to the end of line
Control-u to delete from the cursor to the start of the line

Learn More

Project 48 shows you how to increase the number of commands saved in the command-line history and how to stop sensitive commands from entering it. Project 49 gives you more ways to recall information from the command-line history.

Aliases and Functions

Bash supports the definition of aliases and functions. An alias can be defined to be a complete or partial command line. When it's used as the first word on a command line, Bash replaces it with its definition.

Here's an alias called ll that creates a shortcut for the command ls -al. Enclose the definition in single quotes to ensure that the shell doesn't interpret the definition before it's assigned to the alias.

$ alias ll='ls -al'

Learn More

Project 53 tells you more about command-line editing.

Tip

Project 80 teaches you the black art of shell quoting.

To use the alias, simply type its name and press Return.

$ ll total 280 drwxr-xr-x 45 saruman saruman 1530 May 10 14:13 . drwxrwxr-t 18 root    admin    612 Apr  9 18:22 .. ... drwx------ 13 saruman saruman  442 Apr 14 13:25 Pictures drwxr-xr-x  5 saruman saruman  170 May  4 22:04 Public drwxr-xr-x 25 saruman saruman  850 Apr 16 15:24 Sites

An alias need not form the complete command line. It will accept arguments and option settings that work with the command it's based upon.

Note

A Bash alias is not the same as a Finder alias.

$ ll Sites ...

If you always use a particular command with a specific set of its options turned on, an alias can save typing: Just make the generic command name an alias for the option-enabled version. This technique is used below to make sure the safe -i option is used by default with commands cp, mv, and rm. (See "It's Goodbye, Not Au Revoir" in Project 2 for a recap of why this is a good precaution.)

Type the following lines of code.

$ alias cp='cp -i' $ alias mv='mv -i' $ alias rm='rm -i'

Used without options or arguments, the alias command displays the definitions of aliases you've set up.

To remove an alias you've created, you can use the unalias command. Use it to delete the ll alias you made earlier.

$ unalias ll

Fun with Functions

Functions are similar to aliases, but they can include arguments as well as commands and are often more complex. As an example, a good candidate for use in a function is this command sequence.

$ ls -al <directory> | less

The sequence performs a handy task: It generates a long listing of the specified directory and hands the list off to command less, which ensures that the list is displayed one page at a time without overflowing the length of the Terminal window.

Our new function, which we'll call lll, encapsulates this useful sequence as a custom command, which takes a directory pathname as its argument. After we've defined it, simply typing lll <directory> will display the directory's long listing in less.

Learn More

Project 51 explores Bash aliases at greater length, and Project 52 explores Bash functions.

Type the following carefully to define the function. Don't worry too much about the unfamiliar characters; we'll discuss them at greater length later on. And don't panic when Bash changes its prompt from $ to > when you start a new line; that's its way of saying, "You pressed Return, but I don't have enough information to execute a command, so keep typing."

$ lll() { > ls -al $* | less > } $

List all currently defined functions, using command declare f, and you'll see that Bash now recognizes our new function:

$ declare -f lll () {     ls -al $* | less } $

The final example shows how lll could be defined in a single line: Note that the space after the first brace and the semicolon after less are necessary.

$ lll() { ls -al $* | less;}

To use the function, simply type its name followed by any arguments.

$ lll Sites Documents Library

Make them Permanent

An alias or function is good only for the current shell session. If you start a new shell session, all alias and function definitions are lost. To make them permanent, you must define them during the shell's startup sequence by adding the definitions to one of Bash's startup files. See "Bash Initialization" later in this chapter.

Shell and Environment Variables

Shell and environment variables hold values that can be recalled and used later. Shell variables are local to the current shell session. If you run another command, like ls, bash, or zsh, that command does not inherit (cannot see) shell variables from the current shell session. Environment variables, however, are inherited by new commands, which means they can be seen.

Shell Variables

A shell variable is used to record, and later recall, a value. The command line var='value' sets shell variable var to the value value. Introducing a variable with a dollar symbol, using format $var, causes the shell to expand the variable, replacing it with its value. Here's how to set and use a shell variable called sites.

$ sites='/Users/saruman/Sites' $ ls $sites images          index.html

In the example above, Bash expands the command

$ ls $sites

$ ls /Users/saruman/Sites

Use echo to display the contents of a variable.

$ echo $sites /Users/saruman/Sites

Shell variables are most often used in shell scripts, which are discussed in detail in Chapter 9. A shell variable is visible only to the shell in which it was set. Other commands launched by the shell do not have visibility to those variables.

Use either set or declare to list all shell variables and their values. (The set and declare commands list environment variables too.) To remove a variable, use the unset command. You'll notice that many shell variables are already set, including HOME.

Learn More

Project 80 explores the mysterious world of shell quoting.

You may use existing variables within the declarations for new ones, as in this example, which uses the preset variable HOME to reset the variable sites you just created.

$ sites="$HOME/Sites"

Run the echo command on sites again, and you'll see that it expands the same way, whether it's set using the HOME variable shortcut or the full pathname to the Sites directory. Using preset variables like HOME not only saves typing, but also lets you set variables in a way that will work for all users, no matter what their home directory is.

$ echo $sites /Users/saruman/Sites

The shell expanded $HOME when the variable sites was initialized. Notice the use of double quotes here, versus single quotes in previous examples. Bash does not look inside single quotes to expand variables and therefore would not have expanded $HOME. Bash does look inside double quotes and hence does expand $HOME.

Environment Variables

What differentiates a shell variable from an environment variable is its scope. Whereas shell variables are particular to the defining instance of Bash (or whatever shell you are running), environment variables are part of the environment inherited by all commands launched by the shell. When you set an environment variable in Bash, any command run from that shell (like man or ls) inherits the variable and can read its value. This also applies when you run Bash itself, or zsh, as they are just regular commands too.

Traditionally, environment variables are in uppercase, and user-defined shell variables are in lowercase. The command env lists all environment variables and, unlike set and declare, does not list shell variables or functions.

To turn a shell variable into an environment variable, it must be exported from the shell (made part of the environment). Use either the export command

$ SITES="$HOME/Sites" $export SITES

or the declare command. Option -x says to export the variable just declared.

$ declare -x SITES="$HOME/Sites"

Issue command env to check that SITES has been correctly defined and exported as an environment variable.

$ env TERM_PROGRAM=Apple_Terminal TERM=xterm-color SHELL=/bin/bash SITES=/Users/saruman/Sites ...

LESS is a good example of an environment variable in action. It's read by the command less, where its value is taken as the default list of options that less should assume. The line below will cause less to assume it has been issued as command less -iqM. This is particularly useful when man calls less to display a man page. It gives you a way to influence the way less behaves when called from man.

$ declare -x LESS="-iqM"

Other notable environment variables are PWD, which records the current working directory, and PS1, which holds the prompt. Try this.

$ PS1="What next master? " What next master?

Scope

Scope is best explained by example. In the extract below, we declare and initialize a shell variable (sv) and an environment variable (EV). Then we run a command and examine the environment inherited by the command. In this example, the command is Bash, to make it easy to display the environment, but it could be any command. You'll see that the environment variable is inherited and the shell variable is not.

First, set the two variables, and export EV to make it an environment variable. Note that EV is just a shell variable until it is exported.

$ sv="Shell variable" $ EV="Environment variable" $ export EV $ echo "sv="$sv", and EV="$EV sv=Shell variable, and EV=Environment variable

Next, start a new command (the Bash shell), and display the variables as seen by the new command. We see that shell variable sv loses its value (or, rather, it's not defined in the new shell), whereas environment variable EV is inherited.

$ bash $ echo "sv="$sv", and EV="$EV sv=, and EV=Environment variable $

Next, we change the values of both sv and EV and then exit the command (the new Bash shell), retuning to the original shell. You'll notice that the environment of the original shell is unchangedin particular, EV has its original value. Although a command inherits the environment from its parent, it's free to modify its own environment without affecting that of its parent. Environment variables are inherited but not passed back. This behavior is by design, to prevent a child process from trashing the environment of its parent.

$ sv="Shell variable two" $ EV="Environment variable two" $ export EV $ echo "sv="$sv", and EV="$EV sv=Shell variable two, and EV=Environment variable two $ exit exit

Back in the original shell, we have the original values of sv and EV.

Learn More

Project 47 tells you more about Bash initialization on startup.

$ echo "sv="$sv", and EV="$EV sv=Shell variable, and EV=Environment variable $

Bash vs. Tcsh

The Tcsh shell employs a different syntax for setting shell and environment variables. If you ever use it, here's what you need to know.

To set a shell variable:

% set sv="Shell Variable" % echo $sv Shell Variable %

To set an environment variable:

% setenv EV "Environment Variable" % echo $EV Environment Variable %

Bash Initialization

When a Bash shell starts up, it executes a number of scripts written to set up and customize its environment. Some of these are global and are executed for all users; others are individual to each user. Additionally, Bash can be started as a login shell or a non-login shell, and this affects which scripts are executed. Apple's Terminal application starts a new window with a login shell, while whereas typing bash at the command line starts a non-login shell.

A login shell (opening a new Terminal window starts a login shell) sources (executes) these scripts, and in this order:

/etc/profile This is a global file sourced for all users.
~/.bash_profile Every user has his own version of this.

A non-login shell (typing bash at the command line starts a non-login shell) sources these scripts, and in this order:

/etc/bashrc This is a global file sourced for all users.
~/.bashrc Every user has his own version of this.

Finally, an interactive shell, which is launched to run a shell script and exits when the script completes, does not source any initialization scripts. Thus, it inherits environment variables but does not configure any settings of its own.

Note

Remember that the tilde (~) symbol is used as shorthand for the absolute pathname of your home directory. ~/.bashrc means the file .bashrc in your home directory.

Learn More

Project 9 tells you more about writing simple shell scripts.

What Goes in Which Script?

The question arises as what to put into each of the four script files. Those in /etc are sourced for all users and therefore would include global customization. All users have their own customization scripts in their home directories (those starting with ~/). The login scripts should contain all customization (setting environment variables, shell variables, functions, aliases, and shell options). The non-login scripts should do all this except setting environment variables. Why do the non-login scripts not need to set environment variables? Remember that environment variables are inherited, and so the non-login shell, which is launched from the command line, will inherit its parent's environment variables.

It's possible to avoid the duplication described above (setting shell variables, functions, aliases, and shell options in both login and non-login scripts). Simply add a line in /etc/profile to say, "Now execute /etc/bashrc", and similarly add a line in ~/.bash_profile to say, "Now execute ~/.bashrc". Now the login scripts will simply fetch the appropriate settings from the non-login scripts. (The default /etc/profile supplied with Mac OS X already does this.)

Here's what to do.

Add this line to the end of /etc/profile.

source /etc/bashrc

Add this line to the end of ~/.bash_profile.

source ~/.bashrc

Editing Script Files

Learn More

Projects 6 covers redirection.

The best way to make and change scripts is to use one of the Unix editing tools discussed in Chapter 4. For now, just type along to use the echo and cat commands, along with a technique called redirection, to create some user customization scripts.

The following sequence of commands sets ~/.bash_profile to call ~/.bashrc and defines the aliases described in "Aliases and Functions" earlier in this project.

First, we create a login script in ~/.bash_profile and tell it to execute the non-login script in ~/.bash_rc. Don't worry if ~/.bash_profile already exists; the commands will be added to the end of it.

$ echo 'source ~/.bashrc' >> ~/.bash_profile

Now check the contents of ~/.bash_profile.

$ cat ~/.bash_profile source ~/.bashrc

Next, create the non-login script, and add the commands to define the alias and functions.

$ echo "alias cp='cp -i'" >> ~/.bashrc $ echo "alias mv='mv -i'" >> ~/.bashrc $ echo "alias rm='rm -i'" >> ~/.bashrc $ echo "alias ll='ls -al'" >> ~/.bashrc $ echo 'lll() { ls -al $* | less;}' >> ~/.bashrc

Finally, check the contents of ~/.bashrc.

$ cat ~/.bashrc alias cp='cp -i' alias mv='mv -i' alias rm='rm -i' alias ll='ls -al' lll() { ls -al $* | less;}

Start a new login shell with a new Terminal session (or start a non-login shell by simply typing bash), and verify that the shell is configured as expected.

Check that the aliases and the function exist.

$ alias alias cp='cp -i' alias ll='ls -al' alias mv='mv -i' alias rm='rm -i' $ declare -f lll () {     ls -al $* | less }

Try out the new commands.

$ touch x y $ cp x y overwrite y? (y/n [n]) y $ ll total 56 drwxr-xr-x 20 saruman saruman  680 11 May 10:17 . drwxrwxr-t 18 root    admin    612  9 Apr 18:22 .. ... drwxr-xr-x  5 saruman saruman  170 10 May 10:07 Sites $ lll Sites ...

X11 and xterm

If you use X11 and xterm, be warned that xterm, unlike Apple's Terminal, does not start a login shell. To force a login shell, either invoke xterm with the option -ls or add this line to the file ~/.Xdefaults:

XTerm*.LoginShell: True