Hack 78 FreeBSD from Scratch


figs/expert.gif figs/hack78.gif

For those who prefer to wipe their disks clean before they upgrade their systems.

Have you ever upgraded your system with make world? If you have only one system on your disks, you may run into a problem: if the installworld fails partway through, you may end up with a broken system that might not even boot. It's also possible that the installworld will run smoothly, but the new kernel will not boot.

What if you're like me and believe in the "wipe your disks when upgrading systems" paradigm? Reformatting ensures there is no old cruft left lying around. It also means you have to recompile or reinstall all your ports and packages and then redo all your carefully crafted configuration tweaks.

FreeBSD From Scratch solves all these problems. The strategy is simple: use a running system to install a new system under an empty directory tree, mounting new partitions in that tree as appropriate. Many config files can copy straight across, and mergemaster can take care of those that cannot. You can perform arbitrary post-configuration of the new system from within the old system, up to the point where you can chroot to the new system.

This upgrade has three stages, where each stage either runs a shell script or invokes make:


stage_1.sh

Creates a new bootable system under an empty directory, merges or copies as many files as are necessary, and then boots the new system


stage_2.sh

Installs your desired ports


stage_3.mk

Does post-configuration for software installed in the previous stage

From now on, whenever you feel like an update is in order, simply toggle the partitions you want to wipe and reinstall.

While compiling the ports during stage two, the system will not be available for its usual duties. If you run a production server, consider the downtime caused by stage two. If time is an issue, consider using precompiled packages instead of ports.


8.3.1 Stage One: System Installation

This hack uses several scripts and configuration files that you can download from the original document's site (listed in this hack's Section 8.3.4 section). Also, if you keep your docs up-to-date with cvsup, the scripts and original document can be found in /usr/doc/en_US.ISO8859-1/articles/fbsd-from-scratch.

The script for stage one is stage_1.sh. When run with exactly one argument:

# ./stage_1.sh default

it will read its configuration from stage_1.conf.default and write a log to stage_1.log.default.

You'll need to customize stage_1.conf.default to match your idea of the perfect system. I have tried to comment all of the sections you should adapt. In addition to the customized sections, the configuration script must provide four shell functions:

  • create_file_systems

  • create_etc_fstab

  • copy_files

  • all_remaining_customization

Before you run stage_1.sh, make sure you have completed the usual tasks in preparation for make installworld/installkernel:

  • Configure your kernel config file.

  • Complete make buildworld.

  • Complete make buildkernel KERNCONF=whatever.

The stage_1.sh script will stop at the first command that fails, so you cannot overlook errors. It will also stop if you use an unset environment variable, which is probably due to a typo.

Answer no or press Enter when mergemaster asks if whether should delete /var/tmp/temproot.stage1. This directory contains some files that must be copied to the new system later.

*** Comparison complete Do you wish to delete what is left of /var/tmp/temproot.stage1? [no] no

After that, it will list the files it installed:

*** You chose the automatic install option for files that did not     exist on your system.  The following were installed for you:       /newroot/etc/defaults/rc.conf       ...       /newroot/COPYRIGHT (END)

Type q to quit the pager. Then, you'll have to deal with login.conf:

*** You installed a login.conf file, so make sure that you run     '/usr/bin/cap_mkdb /newroot/etc/login.conf'     to rebuild your login.conf database     Would you like to run it now? y or n [n]

The answer does not matter, since we will run cap_mkdb in either case.

You can download the author's stage_1.conf.default, which you'll need to modify substantially. The comments should give you enough information regarding what to change.

Pay attention to the newfs commands. While you cannot create new filesystems on mounted partitions, the script will happily erase any unmounted partitions. This can be enough to ruin your day, so be sure to modify the device names to match your scenario.

Running this script installs a system that, when booted, provides inherited users and groups, firewalled Internet connectivity over Ethernet and PPP, correct time zone settings and NTP, and more minor configurations, such as /etc/ttys and /etc/inetd.conf.

Other areas of configuration will not work until stage two completes. For example, we have copied files to configure printing and X11. Printing, however, needs applications not found in the base system. Similarly, X11 will not run before we have compiled the server, libraries, and programs.

8.3.2 Stage Two: Ports Installation

It is possible to install precompiled packages at this stage instead of compiling ports. In this case, stage_2.sh will be nothing more than a scripted list of pkg_add commands.

I install my favorite ports via the downloadable stage_2.sh script. You can run it multiple times safely, as it will skip all ports that are already installed. It also supports the dry run option (-n), which will show what would be done. Run it like stage_1.sh, with exactly one argument to denote a config file:

# ./stage_2.sh default

This example will read the list of ports from stage_2.conf.default.

The actual list of ports consists of lines with two or more space-separated words: the category and the port, optionally followed by an installation command that will compile and install the port. By default, this is make install. Most of the time, it suffices to name only the category and port. You can fine-tune some ports by specifying make variables, as found in the port's Makefile:

www mozilla make WITHOUT_MAILNEWS=yes WITHOUT_CHATZILLA=yes install mail procmail make BATCH=yes install

In fact, you can specify arbitrary shell commands, so you are not restricted to simple make invocations:

java linux-sun-jdk14 yes | make install news inn-stable CONFIGURE_ARGS="--enable-uucp-rnews --enable-setgid-inews" \     make install

Note that the line for news/inn-stable includes an example of a one-shot shell variable assignment to CONFIGURE_ARGS. The port's Makefile will use this as an initial value and augment some other essential args.

The difference between specifying a make variable on the command line (as in the last example) and the following:

news inn-stable make CONFIGURE_ARGS="--enable-uucp-rnews \     --enable-setgid-inews" install

is that the latter will override instead of augment.

Be careful that your ports do not use an interactive install; they should not try to read from stdin. If they do, they will read the next line or lines from your list of ports and get confused. If stage_2.sh mysteriously skips a port or stops processing, this is likely the reason.


Finally, this script will create a log file named LOGDIR/category+port for each port it installs.

When you download the stage_2.sh script, you may want to modify these variables at the beginning of the script to reflect your environment:


DBDIR="/var/db/pkg" PORTS="/usr/ports" LOGDIR="/home/root/setup/ports.log"; mkdir -p \     ${LOGDIR}

8.3.3 Stage Three: Post-Configuration

You installed your beloved ports during stage two, but some ports require a little bit of configuration. This is the job of stage three, the post-configuration stage. I have chosen to implement stage three as a Makefile because this allows easy selection of what you want to configure simply by running:

# make -f stage_3.mk target

As with stage_2.sh, make sure you have stage_3.mk available after booting the new system, either by putting it on a shared partition or by copying it somewhere on the new system.

Automating the installation of a port may prove difficult if it is interactive and does not support make BATCH=YES install. For a few ports, the interaction is nothing more than typing yes when asked to accept some license. If such input is read from the standard input, we simply pipe the appropriate answers to the installation command, usually make install. This is how I dealt with java/linux-sun-jdk14 in the previous example.

This strategy, however, does not work for editors/staroffice52, which requires that X11 is running. The installation procedure involves a fair amount of clicking and typing, so it cannot be automated like other ports can. However, the following workaround does the trick for me. First, I created a staroffice package on the old system with:

# cd /usr/ports/editors/staroffice52 # make package =  ==>  Building package for staroffice-5.2_1 Creating package /usr/ports/editors/staroffice52/staroffice-5.2_1.tbz Registering depends:. Creating bzip'd tar ball in '/usr/ports/editors/staroffice52/staroffice-5.2_1.tbz'

During stage two, I used pkg_add to add this package:

# pkg_add /usr/ports/editors/staroffice52/staroffice-5.2_1.tbz

Upgrading Configuration Files

Be aware of upgrade issues for config files. In general, you do not know when and if the format or contents of a config file changes. A new group may be added to /etc/group, or /etc/passwd may gain another field. Simply copying a config file from the old to the new system may be enough most of the time, but in these cases it is not. Unfortunately, mergemaster is available only for base system files, not for anything installed by ports. All you can do is be alert, especially when the major version number bumps. All actively maintained software programs are prime candidates for config file scrutiny. To detect such silent changes, I keep a copy of the modified config files in the same place where I keep stage_3.mk and compare the result with a make rule. For example, I examine Apache's httpd.conf in target config_apache with:

# ... automated httpd.conf modifications here ... @if ! cmp -s /usr/local/etc/apache2/httpd.conf httpd.conf; then \     echo "ATTENTION: the httpd.conf has changed. Please examine if"; \     echo "the modifications are still correct. Here is the diff:"; \     diff -u /usr/local/etc/apache2/httpd.conf httpd.conf; \ fi

If the diff is innocuous, I can make the message go away with cp /usr/local/etc/apache2/httpd.conf httpd.conf. See [Hack #92] for more on this strategy.


The downloadable stage_3.mk will give you an idea of how to automate all reconfiguration.

8.3.4 See Also

  • "FreeBSD From Scratch" (includes links to the scripts) at http://www.freebsd.org/doc/en_US.ISO8859-1/articles/fbsd-from-scratch/article.html



BSD Hacks
BSD Hacks
ISBN: 0596006799
EAN: 2147483647
Year: 2006
Pages: 160
Authors: Lavigne

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net