The Boy Scout and system administrator motto: "Be prepared!"
As a good administrator, you back up on a regular basis and periodically perform a test restore. You create images [Hack #23] of important servers so you can quickly recreate a system that is taken out of commission.
Are you prepared if a system simply refuses to boot?
Some parts of your drives are as important as your data, yet few backup programs back them up. I'm talking about your partition table and your boot blocks. Pretend for a moment that these somehow become corrupted. The good news is that your operating system and all of your data still exist. The bad news is that you can no longer access them.
Fortunately, this is recoverable, but only if you've done some preparatory work before the disaster. Let's see what's required to create an emergency repair kit.
7.4.1 Inventory of the Kit
When you install a system, particularly a server, invest some time preparing for an emergency. On a FreeBSD system, your kit should include:
Place these items in an envelope and store it in a secure location with your backup tapes. Make a note on the envelope of the system to which this kit should apply, along with the version of the operating system. Ideally, you should have two copies of both your emergency kit and backup media. Store the second copy off-site.
7.4.2 Preparing the Floppies
Regardless of how you install a system, take a few minutes to download the *.flp files found in the floppies directory. This is especially important if you use cvsup to upgrade a system, as you can go months or years without the installation CD-ROM or floppy media. Your aim is to test these floppies on your system before a disaster strikes. The last thing you want to be doing in an emergency is scurrying around creating floppies only to find that an essential driver is missing.
Here, I'll connect to the main FreeBSD FTP server and download the files for an i386, 5.1-RELEASE system:
# ftp ftp.freebsd.org Trying 188.8.131.52... Connected to ftp.freebsd.org. <snip banner> 220 Name (ftp.freebsd.org:dlavigne6): anonymous 331 Guest login ok, send your complete e-mail address as password. Password: ftp> cd pub/FreeBSD/releases/i386/5.1-RELEASE/floppies 250 CWD command successful. ftp> binary 200 Type set to I. ftp> mget *.flp mget boot.flp [anpqy?]? a Prompting off for duration of mget. <snip transfer of five files> ftp> bye 221 Goodbye.
I find it convenient to create a floppies directory with subdirectories for each version of FreeBSD I have running in my network. I then download the appropriate *.flp files to the appropriate subdirectory so they are available when I wish to create an emergency repair kit for a new system.
Once you have all five files, you can decide which ones you'll need for your particular system. To perform an emergency repair, you'll need some way to load your version of the operating system into memory so you can access the utilities on the fixit floppy and restore whatever damage has happened to your own operating system. There are several ways to load an operating system.
The first approach is to boot directly from the install CD-ROM, assuming it is bootable and your BIOS supports this. If this is your scenario, you don't need boot.flp, kern.flp, or mfsroot.flp.
If booting from the CD-ROM isn't an option, you can use either boot.flp or both kern.flp and mfsroot.flp. boot.flp is basically the contents of the other two floppies placed onto one floppy. The kicker is that you need a floppy capable of holding 2.88 MB of data.
Depending upon your hardware, you may or may not need drivers.flp. If the installer detected all of your hardware, you won't need this floppy. Otherwise, you will. Finally, if you don't have a CD containing the live filesystem, you'll need fixit.flp, as this floppy contains the actual repair utilities.
Use dd to transfer these files to floppies. Repeat this for each *.flp file you require, using a different floppy for each file:
# dd if=fixit.flp of=/dev/fd0
Label each floppy with its name and version of FreeBSD and write protect the floppies.
7.4.3 The Rest of the Kit
Before testing your floppies, print some important system information you won't remember all of these details in an emergency. First, you'll want a copy of your filesystem layout:
# more /etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/ad0s1b none swap sw 0 0 /dev/ad0s1a / ufs rw 1 1 /dev/ad0s1e /tmp ufs rw 2 2 /dev/ad0s1f /usr ufs rw 2 2 /dev/ad0s1d /var ufs rw 2 2 /dev/acd0 /cdrom cd9660 ro,noauto 0 0 proc /proc procfs rw 0 0 linproc /compat/linux/proc linprocfs rw 0 0 /dev/fd0 /floppy msdos rw,noauto 0 0
Here, I've just sent the output to a pager for viewing. Depending upon how printing is set up on your system, redirect that output either directly to lpr or to a file that you can send to a printer.
Notice that all of my hard drive partitions start with /dev/ad0s1. The name of your hard drive is needed in order to view the partition table, or what FreeBSD calls the disklabel:
# bsdlabel ad0s1 # /dev/ad0s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 524288 0 4.2BSD 2048 16384 32776 b: 1279376 524288 swap c: 30008097 0 unused 0 0 # "raw" part, don't edit d: 524288 1803664 4.2BSD 2048 16384 32776 e: 524288 2327952 4.2BSD 2048 16384 32776 f: 27155857 2852240 4.2BSD 2048 16384 28512
Once you have a printout of your disklabel, complete your kit by printing the contents of /var/run/dmesg.boot. This file contains your startup messages, including the results of the kernel probing your hardware.
7.4.4 Testing the Recovery Media
Now you're ready to test that your kit works before sealing the envelope and sending it off for secure storage. First, boot the system using either your CD-ROM or the emergency floppies. Once the kernel has loaded and probed your hardware, the screen will ask: Would you like to load kernel modules from the driver floppy? If you choose yes, you will be asked to insert the drivers.flp floppy and will be presented with a list of modules to choose from:
cd9660.ko if_awi.ko if_fwe.ko if_sk.ko if_sl.ko if_sn.ko <snip>
Taking a look at those modules, aren't you glad you're testing your kit before an emergency? While the modules don't have the most descriptive names, it's easy to find out what each module represents if you have access to a working system. For example, the modules that begin with if are interfaces. To see what type of interface if_awi.ko is:
% whatis awi awi(4) - AMD PCnetMobile IEEE 802.11 PCMCIA wireless network driver
You can whatis each name; just don't include the beginning if or the trailing .ko. If you do need any of these drivers, save yourself some grief and write yourself a note explaining which drivers to choose off of the drivers.flp. The lucky bloke who has to repair the system will thank you for this bit of homework.
Once you exit from this menu, you'll be prompted to remove the floppy. You'll then be presented with the sysinstall Main Menu screen. Choose Fixit from the menu and insert fixit.flp. You should be prompted to press Alt F4, and you should then see a Good Luck! screen with a Fixit# prompt. Excellent, your floppy is good and your repair kit is complete. Type exit to return to the menu and exit your way out of the install utility.
If this had been an actual emergency, you'd definitely want to read the next hack [Hack #72] .
7.4.5 See Also