Chapter 12: Building Custom Kernels

Rewiring the brain?
Knowing where the parts plug in
makes it possible.

The OpenBSD team strives to produce a good-quality kernel that requires no tweaking beyond setting the occasional IRQ or memory address, or possibly adding another instance of a device driver. The provided kernel works perfectly in almost all circumstances. On extremely rare occasions, you might need to actually build a custom kernel from source. In any event, knowing how the kernel is built can improve your understanding of the system.

The Culture of Kernel Compilation

The open-source world seems to have an ingrained love of recompiling and rebuilding kernels. If you run one of the other free UNIX-like operating systems such as FreeBSD, NetBSD, or Linux, you've almost certainly recompiled kernels. Mailing lists for these operating systems are full of suggestions on rebuilding, tweaking, and modifying the kernel. Kernel reconstruction is a recommended solution to many problems. This is perfectly understandable; you have the source, so use it! In these groups, recompiling the kernel is considered a rite of passage. If you cannot build a custom kernel for one of these operating systems, other users of that OS may even look down on you or tell you it's simple and walk you through it.

If you've come from this sort of background, OpenBSD may be something of a shock. Rebuilding the kernel isn't forbidden, but it's strongly discouraged. The OpenBSD community does not consider building a kernel necessary to prove your adulthood, and rebuilding the kernel is not an appropriate solution to most problems. You will only rarely see an OpenBSD developer suggest that anyone customize the kernel to address an issue, unless there is a serious security issue in the kernel itself. Those who build custom kernels are either masters of the platform, or ignorant newbies. Many people who think they're masters of OpenBSD learn otherwise when they attempt to use a custom kernel.

You've never built a custom kernel for a commercial UNIX such as HP/UX or Solaris, have you? Those kernels are delivered ready to run; while you can tweak them, you shouldn't have to outright replace them. And you might not have even realized that Microsoft operating systems have a kernel, but they do, and no Windows administrator would dream of tweaking it. OpenBSD kernel design leans strongly toward the commercial OS style; the kernel shipped with the product should be good enough for production use.

Suppose you contacted Microsoft about a problem with its latest Windows server operating system and explained that you were having trouble since you recompiled the XP kernel from source, stripping out a few options you simply didn't need and adding in support for experimental features. They would ask you to hold, mute their microphone, and laugh at your request for assistance. You would quite probably be put on speakerphone, just so everyone in the support team could hear what you were saying. You would get no solution to your problem, except perhaps, "Reinstall your system using the kernel provided." On the other hand, you would become a legend; those support people would reminisce about your call for weeks, and eventually you'd wind up as a story told to new tech support people to convince them just how nutty customers really were. ^[1]

Similarly, the OpenBSD project members feel no particular obligation to help users with customized kernels. You're an explorer in dangerous territory and on your own. If your custom kernel crashes, most of them simply won't care. Some developers get very cranky when people demand help with their customized kernels, or when people submit a problem report and don't mention that they're running a modified kernel. Just changing one kernel option might seem trivial, but that one kernel option could represent 20,000 lines of source code that you've just trivially gutted from the kernel. You wouldn't go to your doctor and say, "I'm having trouble when I eat; do you think it's because I ripped out my spleen to run my digestion a little faster?"

The OpenBSD Project is much more friendly than the closed-source vendors, in that they provide the source code for the kernel and give you the tools and instructions necessary to build that kernel. The territory might be dangerous, with rattlesnakes and outlaw hideouts, but they give you a map and a flashlight. If you can carve out some new territory for yourself, good for you!

On the other hand, those same developers are extremely interested in problems with the kernel they ship with the system. If you can demonstrate that a problem exists in the GENERIC kernel, and you can file a complete bug report, they will be all over it. The OpenBSD team cares very deeply about their product. They don't particularly care about what happens when you take their product and rearrange it, which is what you're doing when you customize the kernel.

Why Build a Custom Kernel?

If you're a kernel developer, you might need to build a custom kernel at any time to test new features. In that case, however, you're well aware of what you're doing and thoroughly acquainted with the risks you're taking.

Some people are interested in experimental features within the OpenBSD kernel. Some of these features are available in patchsets, or in OpenBSD-current, or within the existing code as unsupported kernel options. For example the RAIDFrame software RAID feature is not in GENERIC, but is of great interest to some people. Using RAIDFrame requires running a custom kernel - that's why it's not covered in this book. If you're running experimental features, you need to know that "experimental" is a Siamese twin with "unstable."

On occasion, repairing a security flaw requires a kernel patch. These situations are fairly rare, but when they occur you need to be able to rebuild the GENERIC kernel with the patch.

Finally, you might have a machine with a very low amount of memory. Because the OpenBSD kernel (since version 3.1) takes almost 5MB of RAM, you would save memory by reducing the size of the kernel. In most cases this isn't a problem, but if you're running on an old system you might wish to build a stripped-down kernel. This doesn't mean that this usage is approved - while people will certainly understand why you did it, this custom kernel will get no additional support from the community.

Problems Building Custom Kernels

The interdependencies between kernel modules are quite complex and not that well documented. The developers mostly assume that you are willing to read kernel source code and quite a few man pages. It can certainly be done without reading the source code, but you need to be prepared to dig through mailing list archives and websites looking for information on errors you suffer while trying to build your own customized kernel.

OpenBSD's cross-platform design means that the kernel configuration process can be difficult. Different cards run on some busses, don't run on others, and behave oddly on still others. Including the wrong card in your configuration or telling the kernel a card is attached to the wrong bus is a great way to build a non-functional kernel. If you have trouble building a custom kernel due to these complex interdependencies, you need to understand more about how your hardware fits together.

You might have problems building your custom kernel because of a corrupt source tree. Perhaps you applied a patch incorrectly, or you scrambled a file somehow, or you edited the source code in some file. You might try to see if you can compile the GENERIC kernel; if not, you either have a bad source tree or some deeper problem with your system.

If you have trouble building a kernel, feel free to ask for help on <[e-mail]>. State up front that you are trying to build a custom kernel and include your kernel configuration, your version of OpenBSD, the hardware platform you're building it on, and a full description of your problem. Some mailing list reader may be willing to help you out, but they certainly aren't obliged to. (You will probably also get people asking you why you're doing this, much as people might ask, "Why are you cleaning your ears with a power drill?")

Problems Running Custom Kernels

Custom kernels can have all sorts of problems: programs might not run as expected, the system might not boot, or it might not find all of the hardware installed. Most disturbingly, the system might crash, frequently or not, either randomly or reproducibly.

Once you build a custom kernel, the amount of support you can expect from the OpenBSD developers is minimal. The OpenBSD developers spend most of their time on the GENERIC kernel and really aren't interested in errors that happen with customized kernels. If your custom kernel does not work on your system, but the GENERIC kernel does, that means that your custom kernel is broken. Resolving the issue is your problem, not theirs.

If you discover a problem while running a custom kernel, but you can reproduce that problem when your system boots with the GENERIC kernel, the OpenBSD developers will generally be interested. They're interested in just about any way the GENERIC kernel can be tortured beyond endurance. We discuss asking for help and reporting problems in Chapter 1. Be certain to report your problem as occurring on the GENERIC kernel, and include debugging output only from running on the GENERIC kernel. Do not include any information or troubleshooting from when you're running on your custom kernel. It's not only irrelevant, but you may make people think that you're trying to sneak custom kernel debugging past them. (Believe it or not, this sort of thing happens!)

For example, one of my OpenBSD machines is a Pentium 166 with 80MB of RAM. I have a custom kernel in this machine, which enables me to save a couple of megabytes of memory. It hasn't crashed yet, but it could. One day, I might figure out a way to panic the kernel. If I can reproduce this crash on the GENERIC kernel, the OpenBSD team will happily accept a bug report. If I can't reproduce the problem on their kernel, I'm not going to report it. If I debug the problem in depth, and create a patch that fixes the issue, then and only then will I report the problem — and the solution.

^[1]They would also probably ask you where you got the necessary information and source code to rebuild the kernel, but that's a separate issue.