Section 3.9. Backing Up and Restoring with the tar Utility

3.9. Backing Up and Restoring with the tar Utility

tar is the most popular backup utility discussed in this chapter. Many of the files that you download from the Internet are in tar or compressed tar format. One limitation of tar to consider is that it has always had trouble with exceptionally long pathnames. Although it isn't typically used by itself for daily backup and recovery, GNU tar is often used by other open-source tools, such as Amanda (see Chapter 4).

As mentioned earlier, the native version of tar cannot preserve the access times of files that it backs up. If this is important to you, use the GNU version of tar; it can do this.

3.9.1. The Syntax of tar When Backing Up

The basic tar command is as follows:

$ tar [cx]vf device pattern

Now let's look at some example commands. To create an archive of a directory called pattern, use the command:

$ tar cvf device pattern

To do the same thing but with a blocking factor of 20, use the command:

$ tar cvbf 20 device pattern

To do the same thing but have tar verify the data as it writes it (available only in GNU tar),^[**] use the command:

^[**] Yet another reason why you should be using gtar if you are performing regular system backups with tar.

$ gtar cvWbf 20 device pattern

To create an archive of everything in the current directory starting with an "a", use the command:

$ tar cvf device a*

Remember to use the native Mac OS tar if you're running a version later than 10.4. Prior to that, you'll need hfstar.

3.9.2. The Options to the tar Command

tar has two great advantages. The first is the level of acceptance that it has received. The second is its short list of options; there really are not very many:

c

The c option tells tar to create an archive (to make a backup).

v

The v option tells tar to be verbose. It lists the name and size of each file as it is being archived.

W

The W option, available only in GNU tar, tells tar to attempt to verify the files as it writes them.

b blocking-factor

This option tells tar to read and write in blocks of n bytes, where n is the value of the blocking-factor (that you specify) multiplied by the minimum block size (for that operating system). This is normally 512 but could be 1,024. The resulting value, referred to as the block size, can range from 512 to 10,240. A block size of 10,240 would normally mean a blocking factor of 20, because 20 times 512 is 10,240. There is a default value for b if you do not specify it. This default value if usually 20 but could be as little as 1.

f device

This option tells tar to write to the device specified in the device argument, instead of the default tape device for that platform. This device could be a file on disk or optical platter, a tape drive, or standard output (stdout). If you are using GNU tar, it also could be a remote system's tape drive (see the following sidebar "Use GNU tar if You Can"). To send the data to stdout, enter a dash (-) where the device name should be. (Using - is not available on all platforms.)

pattern

This is what generates the include list for tar. Again, it is based on filename expansion syntax, so to back up everything starting with an "a", you enter "a*" as that argument. You can put any filename here, including a directory; this causes everything in that directory to be archived.

Use GNU tar if You Can

GNU tar is an extremely popular utility. Besides being able to read an archive written by any other version of tar, it adds a significant level of functionality. Here are some of its most popular advancements:

The -d option performs a diff compare between the archive and a filesystem. It does this by reading the tape and comparing its contents against the files that it finds in the filesystem. Any differences are reported.

The -a option resets access times (atime).

The -F option runs a script when tar reaches the end of a volume. This can be used to automatically swap volumes with a media changer.

The -Z and -z options automatically pass the archive through compress or gzip, respectively.

The -f option supports remote device names.

By default, GNU tar suppresses a leading slash on absolute pathnames while creating or reading a tar archive. (You can suppress this with the -p option.)

Some people also prefer the GNU style of arguments that are offered by GNU tar. Instead of tar cvf, you can specify tar create verbose file.

GNU tar is available at http://www.gnu.org.

While GNU tar can read an archive created by any other version of tar, the reverse is not necessarily true. Certain native versions of tar cannot read archives created with GNU tar.

3.9.2.1. Listing files on standard input

Most versions of tar do not support listing the files to be archived on standard input, like cpio does. However, GNU tar added this functionality with a T flag that allows you to specify a file that contains a list of files to be backed up. If you want to specify the names of the files to be backed up via standard input, use GNU tar and specify - as the include file. This usually tells it to look at standard input instead of a named file. For example, suppose you wanted to run a find from /home/curtis and back up all the files that you find there:

# cd /home/curtis ; find . -print |tar cvf /dev/rmt/0cbn T -

This causes tar to see the result of the find operation as the list of files to be included.

Some of the native versions of tar that support this feature are listed in Table 3-2.

Table 3-2. Versions of tar that support an include list

tar version	Flag
AIX	-L
DG-UX, SunOS, Solaris	-I
FreeBSD, Linux, GNU tar	-T

3.9.3. Syntax of tar When Restoring

A tar backup is very easy to read. Even if you used a blocking factor when you created the tar, you don't need it for the restore. tar automatically figures it out. (Did I hear you say "How beautiful..."?) To read a backup written with tar, enter:

$ tar xvf device 

or:

$ tar xvf device pattern

The x flag tells it that you are extracting (restoring) from the tar file. The v, f, and device arguments work the same way as they do when making a backup.

3.9.3.1. Restoring selected parts of the archive

When restoring, you can specify the filename(s) that you want to restore by listing one or more pathnames after the device name. It is important to note, however, that the pathname must match the name in the tar archive exactly, or it is not restored. Unlike in cpio, wildcards are not supported in tar. However, if you specify a directory name, everything in that directory is restored. Remember, your specification must match the directory name exactly.

Consider the following example. There is a subdirectory called home, and we create a tar archive of it, called file.tar. You can enter tar cvf file.tar home or tar cvf file.tar ./home. Watch how that affects what you must do to restore from it:

$ tar cvf home.tar ./home a ./home/ 0K a ./home/myfile 0K a ./home/myfile.2 0K

If it was backed up with ./home, it must be restored with ./home:

$ tar xvf home.tar home tar: blocksize = 5 $ tar xvf home.tar ./home tar: blocksize = 5 x ./home, 0 bytes, 0 tape blocks x ./home/myfile, 0 bytes, 0 tape blocks x ./home/myfile.2, 0 bytes, 0 tape blocks

This time it is backed up with home as the pattern:

$ tar cvf home.tar home a home/ 0K a home/myfile 0K a home/myfile.2 0K

Notice again that if it was backed up with home, it must be restored with home. The pattern of . /home does not work:

$ tar xvf home.tar ./home tar: blocksize = 5 $ tar xvf home.tar home tar: blocksize = 5 x home, 0 bytes, 0 tape blocks x home/myfile, 0 bytes, 0 tape blocks x home/myfile.2, 0 bytes, 0 tape blocks

If you don't know the name of the file you want to restore and you don't want to restore the entire archive, you can create a table of contents and look for the file there. First, make a table of contents of the archive:

tar tf device > somefile

If you do that with the archive in the preceding example, you will have a file that looks like this:

home/ home/myfile home/myfile.2

If you knew you were looking for myfile, you could grep for that out of this file:

# grep myfile somefile home/myfile home/myfile.2

You would then know that you should enter:

$ tar xvf device home/myfile

3.9.3.2. Tricking tar into using wildcards during a restore

There is a trick that works most of the time on tape and should work all of the time for tar files on disk. Issue two tar commands at once:

$ tar xvf device \Qtar tf device | grep 'pattern'\Q

If you are using this trick with a tape drive, make sure you use the rewind device, or it won't work! You also might want to add the sleep command to give the tape time to rewind:

$ tar xvf device \Qtar tf device | grep 'pattern' ; sleep 60\Q

3.9.3.3. Changing ownership, permissions, and attributes during a restore

The default actions of tar can vary from system to system, but most versions of tar support the following three options during a restore:

m

Normally, restored files retain the modification times that they had when they were archived. This option changes the modification times to the time of the restore. This is the opposite of its behavior with the cpio command.

tar's default treatment of modification times during a restore is the opposite of cpio's.

o

This option tells tar to make you the owner of any files that you restore. This is the default behavior for users other than root. Unless this option is used, files extracted by root take on the user and group identifiers saved in the tar file.

p

By default, tar normally does not restore all file attributes. File permissions are determined by the current umask instead of the permissions of the original files. Also, the setuid and sticky bits are not restored for any files not owned by the user. This option tells tar to use the permissions of the original files, including any special attributes such as setuid. (You must be root to set the setuid and sticky bits on other users' files.)

3.9.4. Some Other Neat Things About tar

tar has many options, and you should read the manpages to find them all. They can come in very handy.

3.9.4.1. Finding everything that's under the directory

Sometimes things underneath a directory are not what they seem. If you are creating "one last archive" of a directory before deleting it, you might want to follow any symbolic links that you come across. This is what the -h option is for. Make sure you've got lots of tape!

3.9.4.2. Using tar to move a directory

As discussed earlier, cpio has a built-in command to move directories. The problem is that many people do not remember its syntax when the time comes. However, you also can use tar to move a directory. You do this by first cd'ing to one level above the directory you are going to move:

$ cd old-dir ; cd ..

You then use tar and a set of parentheses to create a subshell that "untars" the directory into its new location. (Note the use of the p flag to ensure that tar creates the new directory with the same permissions as the old one.)

$ tar cf - old-dir | (cd new-dir ; cd .. ; tar xvpf - )

The - option for tar cf tells it to send its data to stdout. (We omit the v option to prevent writing the filenames to the display twice.) The - option on the tar xvf tells it to look at stdin for its data. Surrounding the cd old-dir ; tar xvf - with parentheses creates a subshell so that the directory old-dir is extracted into new-dir.

I have seen people try to move a user's home directory by cd'ing into that directory and creating a tar of "*". The problem with this is that it does not include the "." files such as .profile, .cshrc, or .emacs. I have then heard the person say, "Oh, I need to use .*, not *!". Remember always, and never forget, that the expression ".*" matches the string . . (the parent directory). That means the archive also includes the directory above it. That's why it is much easier to go a level above, and tar the directory. (Another way to do this would be to make an archive of ".". I prefer the former because it shows what directory the files came from.)

The syntax may seem a bit difficult, but it is very portable. It could be made a little shorter by saying:

$ cd parent ; tar cf - old-dir | (cd new-parent ; tar xvpf - )

In this example, parent is the directory above the old-dir, and new-parent is the parent directory of the new location. For example, if you were moving /home1/fred to /home2/fred, parent would be /home1, old-dir would be fred, and new-parent would be /home2. Make sure you mean what you type. One of the problems with tar is that you get very familiar with typing tar cvf. Then one day you need to do a tar xvf and accidentally type a c instead of an x. Guess what happens. Your archive is ruined, and there is no way to fix it. This is one of the most common questions on Usenet, and there's never been a good answer for it.

3.9.4.3. Restoring to an alternate location

If you make your tar archives with relative pathnames, restoring to an alternate location is very easy. Simply change directories to something other than the original mount point (e.g., /home1), and start the restore from there. tar creates directories as needed.

If you did not create the tar archive with relative pathnames, you can use GNU tar to take off the leading slash.

Read the cpio section about relative pathnames and why they are important.