11.7 Backing Up and Restoring the System Filesystems

This final section covers backing up and restoring thefilesystem containing the operating system itself, including the case of a system disk failure. Recovering from such a disaster has come to be known as "bare metal recovery" in recent years. Unix Backup and Restore includes detailed chapters describing these techniques and procedures for several Unix varieties.

Filesystems containing operating system files such as / and /usr pose few problems when all you need to restore is the occasional accidentally deleted or otherwise lost file. When the file in question is an unmodified system file, you can usually restore it from the operating system installation media, provided you have it and that it is readable under normal system conditions. If either of these conditions is not true, you should do a full backup of all system filesystems from time to time.

Files that you modify in the system partitions should be backed up regularly. In Chapter 14, we looked at a script that saves all modified configuration and other files to a user filesystem, allowing them to be backed up regularly and automatically via the system backup procedures. Alternatively, the script could save them directly to the backup media (even to a diskette if the archive is small enough).

When system filesystems need to be completely restored (usually due to hardware problems), some special considerations come into play. There are often two distinct approaches that can be taken:

• Reinstalling from the original operating system installation tapes or CDs and then restoring files you have modified. This approach may also involve reconfiguring some subsystems.

• Booting from alternate media and then restoring the filesystems from full backups that you have made.

Which alternative is preferable depends a lot on the characteristics of your particular system: how many files have been customized and how widely they are spread across the various system filesystems, how much device and other reconfiguration needs to be redone, and similar considerations. If you have to restore multiple partitions, it is usually faster to reinstall the operating system from scratch unless unsaved data in another partition on the same disk will be lost using the standard installation procedures.

If you decide to take the second route, booting from alternate media and then restoring from a backup, you will need to make reliable full backups of the system whenever it changes significantly. Because you are depending on them for a system restoration in an emergency, these backups should be verified or even made in duplicate.

In either case, you will sometimes also need to consult records of the disk partitions and associated filesystem layouts, as well as the logical volume configuration, when a logical volume manager is in use. This is vital when the system disk has been damaged and must be replaced to restore the system to its previous configuration. Be sure to keep records of this data (see below).

Here is a general procedure for restoring a key filesystem from a backup (some of the individual steps are discussed in detail Chapter 10):

• Boot off alternate media, either an installation tape or CD, or a special bootable diskette or tape (discussed in a bit). At this point, you will be running off an in-memory filesystem (RAM disk) or one based on the boot medium.

• Create device files for the disks, disk partitions, and/or tape drive that you will need to access, if necessary. They may already have been provided for you if you used a system utility to create the bootable tape or diskette.

• Prepare the hard disk as necessary. This may include formatting (rarely) or partitioning it. Be sure to do anything required to make the disk bootable.

• Create a new filesystem on the appropriate partition, if necessary.

• Mount the system filesystem (/mnt is the conventional location).

• Change the current directory to the mount point. Restore the files from the backup tape. Afterwards, change back to the root directory and dismount the restored filesystem.

• Repeat the process for any additional filesystem and then reboot the system.

There is one additional point to consider when using this approach or planning to rely on it. The filesystem provided by emergency boot tapes or diskettes is very limited, and only a small subset of the normal system commands are available. You will need to verify that the restoration utility you need is available after booting from alternate media. For example, if the boot diskette provides only cpio, the backup of the root filesystem had better not be a tar archive or you will be in trouble. You should also ensure that any shared libraries needed by your desired utility are present. Be sure to verify this before the disaster occurs.

We will now look at this process on each of our Unix operating systems individually.

11.7.1 AIX: mksysb and savevg

AIX provides the mksysb utility for creating bootable backup tapes of the actual live system, which are self-restoring in the event of a failure. It saves all of the filesystems in the root volume group, generally /, /usr, /var, /home (unless you've moved it), and /tmp, plus any paging spaces in rootvg. mksysb is invoked as follows:

# mksysb -i /dev/rmt0

mksysb relies on a data file that records various system configuration information. It is updated by including mksysb's -i option. Use the -m option instead if you wish to restore the exact disk locations of the filesystems in the root volume group as well as their contents (-m says to save the logical volume maps as well as the other configuration information).

To restore the root volume group, boot from the mksysb tape and select the appropriate option from the resulting menu. The system will then be restored from the mksysb tape.

You can use a similar technique to clone a system from a mksysb tape made on a different system. If all the devices are identical, the only restriction is that you should not install a kernel from a multiprocessor system onto a single CPU system or vice versa.

When devices differ between the source and target system, a slightly modified technique is used. First, you boot off the install media, and then you select the option for restoring from a mksysb tape. In this mode, the operating system will automatically substitute drivers from the installation media when the ones on the mksysb tape are not correct for the target system. Note that this method will work only if the target system has the correct drives for accommodating both the mksysb and installation media simultaneously.

11.7.1.1 Restoring individual files from a mksysb tape

mksysb tapes can also serve as nonemergency backups of the root volume group. It is very easy to restore individual files from it. These tapes contain four distinct (tape) files, and the disk files from the root volume group are in the fourth file, which consists of a restore archive.

Thus, you could use the following command to restore the file /usr/bin/csh and the subdirectory /etc/mf from a mksysb backup tape:

# restore -s 4 -x -q -f /dev/rmt0 ./bin/csh ./etc/mf     

The -s option indicates which tape file to use, and the -q option suppresses the initial prompt asking you to press the Enter key after you have mounted the first volume. Use restore's -T option to list the contents of the archive.

11.7.1.2 Saving and restoring AIX user volume groups

The savevg command may be used to back up an entire user volume group, just as mksysb does for the root volume group. For example, the following command saves all of the files in the chemvg volume group to tape drive 1:

# savevg -i chemvg /dev/rmt1

The -i option creates the configuration file needed to save and restore the volume group; using -m instead also saves the logical volume maps, allowing their physical locations on disk to be reproduced.

savevg also has a -e option, which says to exclude the files and directories listed in the file /etc/exclude.vgname from the save set.^[21] Wildcards are not permitted in exclusion lists.

^[21] The mksysb command also recognizes -e, and its exclusion file is /etc/exclude.rootvg.

All of the logical volumes and filesystems and the files within them in a volume group can be restored from a savevg tape; the restvg utility performs this operation. For example, these commands restore the chemvg volume group we just saved:

# restvg -q -f /dev/rmt1 # restvg -q -s -f /dev/rmt1 hdisk4 hdisk5

The first command restores the volume group to its original disks, beginning immediately and without prompting for the first tape volume. The second command restores the structure and contents of the chemvg volume group to disks 4 and 5, shrinking all logical volumes to the minimum size necessary to hold the files within them (-s).

The tape made by savevg is a restore archive, so it is easy to extract individual files from it, as in this example:

# restore -f /dev/rmt1 -T -q # restore -f /dev/rmt1 -x -q -d ./chem/src/h95

The first command lists the contents of the archive, and the second command restores the /chem/src/h95 subtree, creating any necessary subdirectories (-d).

11.7.2 FreeBSD

FreeBSD provides a several options for restoring system files, but all of them require that you have a complete backup of the filesystem from which to restore.

In the event of a system disk or boot failure, you must boot from alternate media (CD-ROM or a boot floppy). Then select the Fixit option from the main menu that appears. At this point, you can choose to boot from the second installation CD (which will function as a live filesystem) or a fixit floppy, or you can start a limited shell. The first two options tend to be the most useful.

The fixit floppy is a limited FreeBSD operating system containing enough tools to restore from a backup. It includes support for the tar and restore commands and tape devices. You create a fixit floppy by mounting the first installation CD and using a command like this one:

# dd if=/cdrom/floppies/fixit of=/dev/rfd0c bs=36b

This floppy can be customized after creation for your specific needs.

In order to save the disk partition layouts on a FreeBSD system, use the fdisk -s and disklabel commands. Along with /etc/fstab, this information will allow you to reconstruct the disk partitions and filesystem layout. The disklabel command can also be used to write a boot block to a replacement system disk.

11.7.3 HP-UX: make_recovery

HP-UX provides the make_recovery facility for creating bootable recovery tapes as part of theIgnite-UX package (the utility is stored in /opt/ignite/bin). A common method of using this utility is the following:

# make_recovery -p -A -d /dev/rmt/1mn  # emacs /var/opt/ignite/recovery/arch.include # make_recovery -r -A -d /dev/rmt/1mn -C

First, we run the command in preview mode (-p). This command does not write any data to tape, but instead creates the file /var/opt/ignite/recovery/arch.include which consists of a list of the items to be included. Here, we are choosing to save the entire root filesystem via -A; the default is to save only the subset of files that are part of the HP-UX operating system.

Once this command completes, we check the /var/opt/ignite/logs/makrec.log1 log file for any errors or warnings. If any are present, we must take any corrective action necessary and then rerun the first command.

Once any warnings are dealt with, the arch.include file can be edited to add or remove items, and then make_recovery can be run again in resume (-r) mode.^[22] The -C option tells the command to update the stored data of the most recent make_recovery procedure.

^[22] In some cases, additional considerations apply when some system files reside outside the root volume group; see the manual page for details.

This process must be repeated after each significant system change. The check_recovery command can be used to determine if make_recovery needs to be run.

Although these tapes are not intended to replace normal backups, it is possible to retrieve individual files from them. To do so, you must manually position the tape to the second file and then extract the desired items with tar:

# cd / # mt -t /dev/rmt/1mn fsf 1 # tar xvf /dev/rmt/1m relative-pathname(s)

The file list should be specified as relative pathnames (e.g., etc/hosts, not /etc/hosts).

The most recent versions of the HP Ignite-UX package also provide make_tape_recovery (creates tape recovery images on the client system itself and from the Ignite-UX server) and make_net_recovery (write a recovery image to the disk drive of the Ignite-UX server across the network). See the documentation for details

11.7.4 Linux

On Linux systems, you can create a boot floppy of the current kernel with this command:

# dd if=/boot/file of=/dev/fd0

Simply copying the compressed kernel to diskette is all that is required, because the Linux kernel is structured so that it is the image of a bootable floppy disk (and it is loadable by either the DOS boot loader or lilo).

This procedure will enable you to boot your system should there be some problem booting from the hard disk. However, if your system disk is damaged and the root filesystem there is inaccessible, you will need a true recovery system to restore things. In such circumstances, you can boot using a rescue disk, which is created with the installation CD mounted with a command like this one:

# dd if=/cdrom/disks/rescue of=/dev/fd0 bs=18k

The rescue floppy contains tools needed to restore a saved backup, including tape devices and the tar command.

To record the disk partitioning information, use the fdisk -l command. Along with /etc/fstab, this information will allow you to reconstruct the disk partitions and filesystem layout, and you can use lilo to create a boot block on a replacement system disk. Note that its -r option will prove very useful when the new root partition is mounted at some other point (e.g., /mnt) within the rescue filesystem.

Recent versions of Red Hat Linux also provide a system rescue option when booting from the installation CD.

11.7.5 Solaris

Solaris provides little in the way of tools for system backup and recovery. You should make full backups of the root filesystem. You can then boot off alternate media to create a minimally working system and restore from your backup.

The prtvtoc command along with /etc/checklist will provide the information required to recreate the disk partitioning and filesystem layout scheme. You can use the installboot command to write a boot block to the system disk. Note thatboot images are stored within the installed filesystem at /usr/platform/model/lib/fs/ufs/bootblk, where model is a string corresponding to your specific Sun hardware model (e.g., SUNW-Sun-Blade-100).

11.7.6 Tru64: btcreate

Tru64 provides the btcreate command for creating a bootable backup tape for the operating system. The tape consists of a bootable miniature operating system and a complete backup of the system files.

Running btcreate is very easy in that it will prompt you for all of the information that it requires. The default (suggested) answers are almost always correct. A restore from a btcreate tape will recreate the logical volume configuration from the original system in addition to restoring all of the system files.

On Tru64 systems, you can use the disklabel -r command to record disk partitioning information and recreate them if necessary.