Basic Unix Commands

Unix has literally hundreds of standard commands. A basic design philosophy of Unix is to have small commands that do one thing right. For example, a Macintosh Application may be called Super File Friend. Our imaginary application can do all types of things with files: it can duplicate a file, it can rename a file, it can delete a file, and it can even read a file! In Unix, all of these functions are possible via the use of separate programs: cp, mv, rm, cat. All of these small commands can be combined to accomplish any desired result because the small, specialized programs provide an incredible degree of flexibility.

Unix command syntax

The basic form of a Unix command is

command-name switches arguments

The command-name is the Unix name of the command, such as ls or mv. Switches are the options you can specify to modify the default behavior of the command and are usually preceded by a minus sign (-). Arguments are strings (frequently, but not always, file names) that provide the command’s input and may also specify the output destination.

Unix commands are the CLI equivalent of Mac OS X applications and menu selections. Although Unix commands can be placed anywhere you have permission to access and execute them, traditional organization has them in one of the directories specified by the PATH environment variable. (Type echo $PATH in the Terminal to see what directories, separated by colons, are automatically searched for commands.) If the command is not located in one of these directories, its location must be fully specified for the shell to execute the command.

A traditional Unix file system is case-sensitive in the naming of files and directories — that is, the file INSTALL is different from the file Install, which is also different from the file install. Mac OS X modifies this behavior so that file names are not case-sensitive in the GUI, but case-sensitivity is still the rule when you are working in the shell.

Note

Although Mac OS X eliminates case-sensitivity from file names, it does not eliminate case-sensitivity from switches or built-in commands. You must still enter them in the proper case.

Log in and log out

Remember, Unix is a multiuser system. Even if you are the only person who uses the computer, it’s still multiuser. As the Process Viewer application shows you (refer to Chapter 22), a user named root executes all sorts of system-level processes, even when you’re the actual person using the computer.

Specifying File Names

The Macintosh file system is a hierarchy in that you use folders to contain files and other folders, and this process can be carried to an arbitrary depth. You can consider each volume to be a separate hierarchy, or you can consider the Desktop as being the top of the hierarchy, containing the various volumes.

With one significant alteration, Unix takes a Desktop View of your file hierarchy. Viewed as an upside-down tree, the top of the Unix hierarchy is called by a single-character name, /, called the root of the tree. The one difference from the Desktop analogy is that in a Unix file system, the mount points are not necessarily mounted directly below the root level — they can reside within folders.

The slash character is also used to separate levels in the path from root (or any other reference point) to a given file. For example, the full path to a file named Diary.rtf in the Documents folder of a user named Sam would be:

/Users/Sam/Documents/Diary.rtf

If you omit the leading slash, you are assumed to be specifying a path relative to your current directory rather than an absolute path specification.

When you first installed Mac OS X, you created an administrator account and gave it a password. To make the process easier for you, this account logs in automatically when you start your computer; however, you can turn that off in the Login pane of System Preferences (refer to Chapter 13) if your computer is going to have multiple users because you might not want your business partner to be logged in as you when he or she restarts the computer.

Just as you are always logged in as a specific user when using the Mac OS X GUI, you are also logged in as a specific user when you use the Terminal application — the CLI. One difference is that you can use Unix commands to change which user identity is in effect for part of your Terminal session. The command used to change users is called su (for switch user). Until you create other accounts or enable root login in NetInfo Manager, no other identities are available to you.

Tip

You can execute a single command as root, even if you haven’t enabled the root login by using the sudo command followed by the desired command as an argument to sudo. For example, sudo chown root myfile executes the chown (change owner) command, causing the file, myfile, to belong to root. As a caution, you will be prompted for your own password when using a command through sudo. Access to sudo is configured by a text file etc/sudoers. This file is generally manipulated via a program called visudo. Visudo is a wrapper application, which opens the /etc/sudoers file for editing, but in addition it locks the file so it can not be changed by the system while being edited. For more information on sudo and visudo, including configuring sudo for multiple users consult man sudo, man visudo and man sudoers.

The most visible effect of each user having a distinct identity is that, when each user creates files, the files are marked as belonging to that user. You can see this in the Finder by choosing File Show Info and then choosing Privileges in the Info window’s pop-up menu. In Terminal, type ls -l thefile (assuming thefile is the file’s name).

As shown in Figure 24-1, the default command line includes your username. If you’ve changed your prompt or are unsure or curious, the whoami command tells you your username.

One big advantage in using Terminal to execute a few commands as another user is that you don’t have to log out of the system to run commands as another user. Although Panther offers Fast User Switching which will allow you to keep your session open while you log in as another user to execute a command. You can just su to that user, as shown in Figure 24-2, execute the commands, and terminate their shell session with exit or Control-D. This may be faster than changing users in the GUI.

Figure 24-2: Use the su command to temporarily change your identity.

Managing files and directories

Just as you always have an active window in the Finder or a current folder in which an application opens or saves files, you have a current directory (also known as working directory) when using the Terminal application. As you can see in Figures 24-1 and 24-2, the default shell prompt provides your current directory and user account name. However, you can customize the shell prompt, possibly to make it shorter and still find out your working directory by entering the pwd (print working directory) command.

As was noted earlier in this Chapter, some shortcut symbols are used with the shell. In particular, shortcut symbols for directories are ~ (user’s home directory), . (current directory), and .. (parent directory to the current directory).

To change your current directory, you use the cd (change directory) command. For example, cd ~ takes you back to your home directory, and cd / takes you to the root of the file system.

Obtaining a list of the files in a directory is as simple as entering the ls command. This command has a number of switches available that you can use to modify its behavior; the more useful and common ones are described in the following list:

-a	Show all files, even invisible (those whose names start with a .) ones.
-F	Append a character to the names of executables (*), directories (/), and links or aliases (@). Characters for sockets are (=), whiteouts (%), and queues (|), which we aren’t going to get into here.
-f	Don’t sort the output (default is to sort alphabetically).
-L	If the file is a link, resolve the link and list that file.
-l	Long listing, including owner, group, size, permissions.
-n	Use the user and group ID numbers rather than names in a long listing.
-R	Recursively list all subdirectories.
-r	Reverse the sorting order.
-S	Sort by size, largest first.
-s	List the number of 512-byte blocks actually used by each file.
-t	Sort by time modified, most recent first.
-x	Sort multicolumn output across page rather than in columns.
-1	Force output to one item per line (screen output defaults to multicolumn).

Another useful command when dealing with files and directories is the file command. If you type file /Users, the file command attempts to tell you what kind of file /Users is — in this case, a directory. The file command also recognizes other file types, such as TIFF, RTF, text, and so forth.

Using Unix to delete stubborn files

Sometimes Finder is unable to move or delete files in the trash. The error message “The operation cannot be completed because you do not have sufficient privileges for some of the items” is displayed. In this event, you can use two options to remove the file using Terminal. The first way is to use rm. Use the following steps to try both options:

First, try simply removing the file, using sudo to have root privileges used for the operation.

1. Type sudo rm filename (or sudo rm -rf foldername if it’s a folder). You are then prompted for your password by sudo.

2. Type the password, and the prompt should return. If there is no error, then the file has been removed.

If that fails, it’s due to a special file flag which has been set on the file, which marks it to the system as locked or protected.

First, you remove the flag and then you remove the file.

1. Type sudo chflags -R nouchg filename. After taking your password, the prompt will return.

2. Type sudo rm filename to remove the file. If you simply want to place the file in the trash and delete it later, you can now use the finder to put the file in the trash.

Autocompletion of file names

One of the really nice features of bash and tcsh is that you don’t always have to type long filenames. Just as Internet Explorer attempts to complete URLs as you type, these shells attempt to complete the name for you if you press the Tab key while typing a file name. If you have only one choice that completes what you have typed thus far, the shell completes the name; however, if multiple possibilities exist, a list will be printed (as shown in Figure 24-3), and the terminal bell will beep, waiting for you to type more characters and to then press Tab again. This process continues until your typing prior to pressing the Tab uniquely identifies a file or, using wildcards as described shortly, a group of files.

Figure 24-3: Desktop and Documents are offered as possible completions for the letter D as entered. To complete either choice, continue typing either word, then hit tab as soon as it becomes unique.

Unix wildcards and regular expressions

You’re probably getting the idea by now that one of Unix’s characteristics is to express the most information in the tersest manner. Using regular expressions, a shorthand notation for arbitrary strings of characters, the shell buttresses this impression. If you’ve used BBEdit (BareBones Software, www.barebones.com) or any of a number of other applications, you’ve encountered grep (global regular expression parser), and the power that this shorthand provides in matching patterns and strings.

Regular expressions can be used on the command line in arguments to a command or within applications (particularly editors) to perform searches or find and replace operations.

In its simplest form, an expression is a string of characters, such as document name. Two special wildcards are used in regular expressions:

• Asterisk (*), which stands for any sequence of characters.

• Question mark (?), which stands for any individual character.

You can also tell the shell to match any character from a list by enclosing the list in square brackets; for example, [aeiou] would indicate that any lowercase vowel would be a matching character. You can even tell the shell to match any character that is not in the list by preceding the list with a caret (^). Additionally, special characters are available if you want it to match at the beginning of a line (^) or end of a line ($).

Note

Yes, the caret is used for both negation and to denote beginning-of-line. Here, you need to be aware of the context. If the caret is the first character within square brackets (a set of characters), it means “anything except the characters enumerated.” But, if the caret is outside the brackets and is the first character of the expression, it means “beginning of line.”

Giving you even more to remember (but less to type), you can specify ranges of characters, for example [a-z] to specify any lowercase alphabetic character. Now, the obvious question arises, "But what if I want to match a hyphen?" The so-called escape character (\) comes into play here — any character following the escape character is to be taken literally, to specify a backslash and to escape it as well. In other words, an \ character is used to escape any special character and to hide it’s meaning from the shell. In the case that you want to use a \ character literally you’ll escape it as well by using the \ character as in \\.

Tip

Certain escaped characters have special meaning in regular expressions. To denote a line break, you use \n. To specify a tab character, use \t; and to match a page break, type \f.

You can use metacharacters to specify how many times a pattern may repeat. The pattern may be a literal character, a wildcard character, a character class, or a special character. The asterisk (*) denotes zero or more occurrences of the pattern — therefore, the pattern is always true. Similarly, the question mark (?) signifies zero or one occurrence of the pattern and also indicates a match for every character scanned. Finally, the plus sign (+) tells the shell to find one or more occurrences of the pattern.

Note

There is actually another repetition indicator. You can enter a {n}, where n is a digit, to indicate matching exactly n occurrences of the pattern. Entering {n,} specifies matching n or more occurrences of the pattern, and entering {n,m} indicates matching at least n but no more than m occurrences of the pattern.

Just as you can combine patterns to form more-complex patterns, you are provided with an alternate character (an or operator), enabling you to match any of a collection of patterns. This character is the vertical bar (|).

Table 24-1 illustrates some of the ways that you can use wildcards and regular expressions to find matches.

Table 24-1: Use of Wildcards and Regular Expressions to Find Matches

Pattern	Meaning
[Ff]ile[0-9]	Match anything spelled file, whether or not the f is capitalized and followed by a single digit.
*.rtf	Match any string ending in .rtf.
^From*Spenser$	Find ny line starting with From and ending with Spenser.
^[^a-z]	Find the first character of any line that does not start with a lowercase letter.
1[01]:##[\t][Pp]\.[Mm]\.	Find any time entry starting at 10 p.m., but before midnight.
tom|dick|harry	Find any tom, dick, or harry (not on the command line; only in grep, Unix’s global regular expression parser command).

You can even create remembered patterns by enclosing the pattern within parentheses. These remembered patterns are often referred to as tagged regular expressions. Within the same command, you can specify the remembered patterns by specifying \1 for the first, \2 for the second, and so forth in a subsequent argument. Such references are referred to as back references. Back references can take considerable time to evaluate; however, they are of particular advantage when doing search-and-replace operations in an editor.

Note

Operators function in precedence (order) in regular expressions. Repetition operators are evaluated before concatenation operators, and concatenation takes precedence over alternation.

Creating and deleting directories and files

Analogous to the Finder’s New Folder command, the shell offers you the mkdir command to create a new directory. You can even create a new (empty) file by using the touch command and specifying a file that does not already exist. The touch command on an existing file changes the file’s modification date and time to the current date and time.

Unix doesn’t really need a rename command, because it has something just as good; the mv (move) command. If you move a file with a new name rather than copy it into the directory in which it already resides, you have renamed it. Moving a file deletes the original. Of course, you use a cp (copy) command when you don’t want to delete the original. Both commands can take one of two forms. They have two arguments (source and destination file names) or they have multiple arguments, the last of which is the directory into which all the other arguments are moved or copied.

Both cp and mv have a number of switches available, and you can read about them by typing man cp or man mv in a Terminal window. The type that we’re going to tell you about here is the one that you’re probably going to use the most. The –R switch tells the shell to recursively copy or move the contents when a source argument is a directory.

You delete files with the rm (remove) command. Unix shows a different mindset from that typical of the Mac OS or even Windows; the effects of rm are immediate and irrevocable. If you want to provide a little bit of safety while possibly increasing the annoyance factor, you can use the –i option, which interrogates you on every file that the command is going to delete, as shown in Figure 24-4. A y is an affirmative response. Anything else is taken as a negative response.

Figure 24-4: Using the -i switch with rm.

To remove an empty directory, you use the rmdir command. If the directory isn’t empty, you receive an error message to that effect, and the directory is not deleted. You can, however, use the rm command with either the -R or -r switch to recursively delete a directory and its contents. Again, adding the –i switch causes the shell to interrogate you for each file and directory before deleting it.

Disk and file system statistics

Two commands, du (disk utilization) and df (display free space), let you find out how much space you are using for your files and directories and how much space is available for use.

Displaying free space

By default, the df command gives you a report such as the one shown in Figure 24-5. It gives you a list of all the file systems you have mounted, where they are mounted, how many 512-byte blocks are on the file system, how many are used, how many are still available, and the percentage of capacity utilized. Divide the number of blocks by 2 to get the number of kilobytes or use the –k option to specify that you want the results in kilobytes.

Figure 24-5: df tells you how much space each file system has and how much is being used.

You can give a single file as an argument to df to get statistics for just the file system on which the specified file resides. Notice that the file system names are not normally the names of the volume (such as Macintosh HD) but rather the cryptic device names.

Disk utilization

The du command breaks the information down into finer increments, giving you statistics for files and directories, as shown in Figure 24-6. With no argument(s), du reports on utilization by all files and directories, recursively, from the current directory. With a file for an argument, you get the number of 512-byte blocks used by that file. With a directory as an argument, du gives you the information for that directory and recurses through any directories contained within the directory (see Figure 24-6). Employing the –s option tells du to summarize (not enumerate), also shown in Figure 24-6.

Figure 24-6: To get an itemized breakdown of disk space used, use df without options — the –s option gives a summary. The output shows 40 512-byte blocks in use by the iTunes directory, or 20k of disk space used.