Rebuilding Your System from Sources

You're ready to go. The rebuilding process consists of a good number of steps, so take your time with it, especially if this is your first time through.

The make world process itself consists of four main compilation steps, listed in Table 19.4. If you're feeling exceptionally confident, you can reduce the process to two steps, but for this first time, take each of the make world steps individually.

Table 19.4. Steps Involved in a Complete make world Process

Four-step Command	Two-step Command	Meaning
make buildworld	make world installworld	Builds and installs make everything but the kernel
make buildkernel	make kernel installkernel	Builds and installs make the kernel

The make world and make kernel commands should be used only by seasoned veterans who really know what they're doing; everyone else needs to use the four individual make commands, which is a far safer way to go and barely any less convenient. You won't be using those commands in the order shown in the table, though! You actually build the world first, build and install the kernel, reboot into single user mode, and then install the rest of the world and merge your configuration files. The following sections detail the steps for this process.

Cleaning Out /usr/obj

This step isn't necessary if this is the first time you've done a make world. If it isn't, you'll probably want to clean out everything in /usr/obj to speed things up and prevent collisions when the system runs into files it can't overwrite.

/usr/obj is where the object files (compiled components) are stored after being compiled and before being installed. Deleting them, however, isn't necessarily as easy as simply doing an rm -rf inside that directory. Making the world creates certain files with the schg flag (the "system immutable" flag), which means that the files can't be deleted, even if you're root. This is a safety measure intended only to provide an extra means of protection against accidents (such as rm -rf \.*). To get around this and clean out the build directory properly, use the following commands:

# cd /usr/obj # chflags -R noschg * # rm -rf *

You won't be able to do this so easily if you're running an older FreeBSD installation and securelevel is set to 1 or higher (this is a systemwide security setting you select during installation or in /etc/rc.conf, as described in man securelevel). This setting prevents you from unsetting the schg flags. You have to reboot into single user mode to complete this step properly.

Starting an Output Log

Keep a log of the output from your make world. If anything goes wrong, you can review the log for clues and, if necessary, post the relevant parts of it to the appropriate mailing list. Chances are that because an error will cause the compile to fail immediately, you'll have the relevant lines in your terminal program's scrollback buffer. However, if you're doing the rebuild from the machine's local console, you won't have a scrollback buffer to copy from, and so it's a good idea to have a transcript of the entire process to peruse if you need to.

Use the script command to create the log. This command effectively runs a shell within a shell, capturing all output into the file you choose. You remain in the script shell until you type exit to return to your regular shell. Here's an example showing the make buildworld command issued after a script log has been started:

# script ~/buildworld.out Script started, output file is ~/buildworld.out # make buildworld ... # exit Script done, output file is ~/buildworld.txt

The argument to script specifies the target file that will contain all the build output. It's a good idea to name this file according to the build step you're about to capture. Each time you run script with the same target filename it overwrites that file rather than appending to it. Therefore, you should run script separatelyand have a separate target filefor each step in the make world process.

Caution

Don't specify a target file in /tmp! Files in /tmp are deleted on bootup, so you won't have your output files when the system comes back up.

Running and Troubleshooting make buildworld

Here's where the fun really begins. Go to /usr/src, take a final look around to make sure everything looks okay, and enter the first make step:

# make buildworld

Now, sit back and watch. The buildworld process takes an hour or two, depending on your hardware. You can speed it up by using the -j option to run multiple simultaneous processes, as in this example, which starts four of these parallel tasks:

# make -j4 buildworld

This is useful even on a single-CPU system, but if you have multiple processors, you can get still more performance from higher values (try as many as 10).

The build process takes a vaguely alphabetical, recursive path through /usr/src. You can track its progress by looking through the directories in /usr/obj. When you see it compiling things in /usr/src/usr.sbin, you know you're near the end.

As is the case with CVSup, the process of building a complete operating system is probably an unfamiliar one, and you can expect to run into some problems. Here are two of the most commonly reported problems and their solutions:

• The compiler fails with "signal 11" errors.

  Signal 11 refers to a segmentation fault in the compiler, which most frequently occurs due to hardware issues. Check to see whether your CPU is overclocked; this can frequently cause problems with FreeBSD when you perform processor-intensive tasks such as compiling software. If you're not overclocking, suspect your RAM or other hardware. Consider memtest86 (http://www.memtest86.com/) as a freely available method for testing your RAM.

• The compiler fails with a lot of *** Error code 1 *** lines.

  This is the generic error when the build fails at any certain point. The buildworld process won't proceed if any part of it fails, so the last few lines of output leading up to the failure are the most useful.

• An error occurred that seems to be related to incorrect compiler flags, or to some mismatched configuration file in /etc.

  Certain files, such as /etc/make.conf, affect the buildworld process and can make the difference between success and failure. A tool that you'll be using later, mergemaster, can help here: run mergemaster -p ("pre-buildworld mode") to compare the critical files to the ones in the new source tree, and install or merge the newer ones if necessary. See the "Running mergemaster" section a little later for a full discussion of how mergemaster works.

To avoid compiler failures resulting from incompatibilities between old object files and new sources, make sure you have completely removed the contents of /usr/obj before compiling. Leaving the objects from previous builds intact can save time, but it can also cause spurious failures if the compiler doesn't realize it has to rebuild some component. Clean out /usr/obj, as shown earlier, and try again.

If this doesn't help, search the mailing list archives at http://www.freebsd.org (in the mailing list that pertains to your code branchstable/freebsd-stable or current/ freebsd-current) on the relevant keywords from your output. If you find nothing useful, post a question to the relevant mailing list, citing the output of uname -a, the time of your last CVSup, and the relevant final few lines from the compiler output.

Another possible recourse would be the #freebsd or #freebsdhelp channels on various IRC servers. See Appendix D, "Sources for More Information," for fuller details on available IRC channels.

Upgrading the Kernel

Okay, your "userland" (user-accessible files, or everything but the kernel) is now built and ready to be installed. First, though, you need to get the new kernel built and installed.

Upgrading a GENERIC Kernel

If you're running a GENERIC kernel (see Chapter 18, "Kernel Configuration," for details), the process is simple:

# make buildkernel # make installkernel

Or, even more simply (though dangerously):

# make kernel

Now, your new kernel is installed as a directory full of modules called /boot/kernel (a single binary called /kernel prior to FreeBSD 5.0), and the old one is at /boot/kernel.old. As discussed in Chapter 14, "System Configuration and Startup Scripts," you can boot with the old one if the new one fails.

Upgrading a Custom Kernel

The upgrading process is made a bit more complex if you're running a custom kernel. The adventurous might try building a new version of the custom kernel right away and running with that, but you can set up a safety net by building a GENERIC kernel first and keeping it on hand to boot with if you have to. You'll also have your old kernel (the one currently running) as a fallback if all else fails.

Still within /usr/src, enter the following:

# make buildkernel KERNCONF=GENERIC

This is your first backup kernel, a GENERIC one built from the new sources. Now, if you haven't done this already, copy /boot/kernel to /boot/kernel.prev (or some other name to indicate that this is the kernel you were already running and you know works). Next, install the newly built kernel:

# make installkernel KERNCONF=GENERIC

Then, duplicate this kernel and call the copy kernel.GENERIC:

# cp -R /boot/kernel /boot/kernel.GENERIC

Now, here's how to build your first-choice kernel (the custom one built from the new sources):

# make buildkernel KERNCONF=CUSTOM

Naturally, replace CUSTOM with whatever the name of your custom kernel is. Now, install the new custom kernel as /boot/kernel:

# make installkernel KERNCONF=CUSTOM

To recap, your lineup of kernels is shown in Table 19.5.

Table 19.5. Newly Available Kernels

Kernel	Description
/boot/kernel	New custom kernel
/boot/kernel.GENERIC	New GENERIC kernel
/boot/kernel.prev	The old tried-and-true kernel

When you reboot, if the first kernel fails, try booting with the new GENERIC kernel. As described in "Understanding the FreeBSD Startup Process" in Chapter 14, this is accomplished by selecting option 6 (Escape to Loader Prompt) in the loader menu, and then entering boot kernel.GENERIC. If that fails, boot with the previously working kernel, and you'll be back to where you started.

Troubleshooting the Kernel Upgrade and Installation

Building and installing the kernel is the riskiest stage yet in the make world process, and the safeguards in the system that protect the kernel during this process add to the arcane nature of the procedure by being quite user-unfriendly. Here are two few common failure modes and their solutions:

• The compiler fails.

  The kernel sources can fail to build for the same reasons that building the rest of the system can fail. If you can't resolve the failure yourself using the same methods as with the buildworld process, try the mailing lists.

• The system won't let me install my new kernel!

  If you're on an older system (before securelevels were deprecated) and have set securelevel to 1 or higher, make installkernel won't be able to remove the schg flag on the kernel. You will have to reboot into single user mode to complete each step of the process. Remember that when you boot into single user mode (using option 4 in the loader menu), none of the filesystems will mount automatically; you'll need to run mount -a before you can go into /usr/src and run the make installkernel process.

Running and Troubleshooting make installworld

After you've completed your kernels, it's time to reboot into single user mode, which prevents multiuser processes (either run by other users or by daemons operating automatically) from altering files that you'll be upgrading. Collisions of this type can lead to serious instability. Using single user mode is also a good idea simply because of the slight speed advantage and because it's nice to be able to bring the system back online completely upgraded and ready to go.

Reboot into single user mode by using the reboot command. When you reach the loader menu, select option 4 (Boot FreeBSD in Single User Mode) to bring the system up far enough for you to verify that the new kernel is working and to complete the installation.

Note

If the system fails to come up (it might crash, it might spew error messages, or there might be any number of different failure modes), first try rebooting normally to see if the behavior is different from case to case, and to try to determine what in the new kernel is causing problems. If you still have no luck, it's time to drop back to your old kernels. Reboot again, and try booting single-user with the backup kernel, as you saw a little earlier. Keep trying kernels until you reach the one you were using before you began the upgrade (which should boot without troubleso far, the kernel is the only thing you've changed in the system). You should then postpone your make world and try again from scratch after you've resolved the problem with your new kernel. Chances are that a kernel failure at this point is indicative of a more widespread and temporary problem that will be resolved in the source in the near future.

In single user mode, you're root by default. The first thing to do is to mount your filesystems, so you can have access to the source tree; do this with a mount -a command and then verify that it worked by using df (all your filesystems should be listed in the output). Go back into /usr/src now and enter the final, most important part of the make world process. Installing the system binaries won't take as long as building them, but there is an equal chance of errors occurringand this time, a failure will result in a partially altered system. Make sure to start your script output log before you begin this step. Here's the command for this step:

# make installworld

After this step completes, your binaries and kernel should be compatible; try running utilities such as ps and top to make sure they work properly. You'll get errors if the kernel and the binaries aren't built from the same source base, so if the utilities work, you can be pretty confident you successfully installed everything so far.

Anything that goes wrong during the make installworld step is potentially dangerousyou can end up with a partially installed system, with some of it compatible with the new kernel and some of it not. Fortunately, there are fewer things that can potentially go wrong than in the other stages. Here are the most common problems:

• The installer fails, citing a problem with permissions or ownership.

  This is the reason for synchronizing your /etc/group and /etc/master.passwd files before installing the world. Check again to make sure some user or group isn't missing and try again.

• An error occurred that seems to be related to incorrect compiler flags, or to some mismatched configuration file in /etc.

  Certain files, such as /etc/make.conf, affect the installworld process and can make the difference between success and failure. A tool that you'll be using later, mergemaster, can help here: run mergemaster -p ("pre-installworld mode") to compare the critical files to the ones in the new source tree, and install or merge the newer ones if necessary. See the "Running mergemaster" section a little later for a full discussion of how mergemaster works.