2.6. OS Tuning

Both FreeBSD and OpenBSD have a variety of options to help you tune the operating system to support the specific needs of your application. As with most BSD configurations, a great deal of the pertinent options are set with sysctl options in /etc/sysctl.conf. Some FreeBSD options available at boot time are set through /boot/defaults/loader.conf. OpenBSD is less flexible than FreeBSD in this regard. Some options can only be set or modified by adjusting your kernel configuration and recompiling your kernel from source.

To Tweak or Not to Tweak

There may come a time, whether you are using FreeBSD or OpenBSD, that your current server and its configuration just do not give you enough performance to meet your needs. You may want faster response time, higher overall throughput, or a higher maximum concurrent connection count. There are definitely options that you can tweak to get more performance from a server. However, they are likely to gain only incremental improvements. Will you double your throughput or triple your concurrent connections? Possibly, but not very likely unless your starting configuration is abysmal.

If you're looking for small increments of improvement, consider the tuning options presented in this section. Some specialized servers (e.g., a busy VPN concentrator) may need some simple tweaking to increase kernel resources dedicated to networking. However, if you are looking for substantial leaps in capacity, consider whether your budget for equipment and staff time can support adding one or more redundant servers. You can spend a lot of time fine-tuning a server, only to eke out a small incremental gain in the end.

It is worth mentioning, especially in the case of FreeBSD, that most of these options automatically tune themselves at boot time. The kernel is aware of your RAM, CPU speed, and total virtual memory. It scales many values based on predefined, carefully worked formulas. All the FreeBSD kernel developers recommend that you avoid tuning these variables unless you really understand the interdependencies and implications. The OpenBSD developers even discourage rebuilding your kernel, because you can create configurations that they have not tested and find yourself using an unstable kernel. If you still really want to get under the hood and tune, read on.

2.6.1. maxusers: Basic Influence

In the very old days, the kernel's maxusers variable was one of the only readily accessible variables. By tuning it, you would set maximum values for a wide variety of filesystem resources, network resources, and other in-kernel data structures. It was statically compiled in and required rebuilding the kernel to change. In OpenBSD, it is still done this way.

FreeBSD recently moved to a dynamically set maxusers variable by default. When it is left unchanged (set to zero), a variety of formulas are used at boot time, based on physical RAM, to determine how large most data structures should be. Generally speaking these calculations are probably right for you. But you can explicitly define maxusers to be something nonzero in your kernel configuration if you are not happy with the automatic calculations.

2.6.2. Increasing Maximum Values

If your server is very busy, you may find that it cannot open enough files to serve all the requests being made. This is an easy issue to diagnose because the error messages are logged to the kernel facility via syslog and are quite clear. Set the variable kern.maxfiles to a number that is large enough, but be careful. Tune it gradually, and only to a value high enough to stay above your high-water mark.

You may also want to tune the maximum number of inbound TCP connections you can handle. In FreeBSD, the variable that controls this threshold is kern.ipc.somaxconn, whereas in OpenBSD it is kern.somaxconn. Tuning this value high will increase the memory used by the kernel data structures associated with network connections. Tuning it too high can reduce performance. Consider carefully before tuning it. If possible, evaluate your changes in your test environment before putting them into production.

2.6.3. Network Buffering

You can also tune the amount of memory available to buffer data going in and out of the network subsystem in the kernel with net.inet.tcp.sendspace. This controls the buffer size available to outbound network flows; increasing it allows more data to be queued up, decreasing context switches to fetch more data from disk, allowing more efficient sending of data. If you predominantly send data in small chunks (e.g., a dedicated DNS server), increasing the size of this buffer will have no beneficial effect. On the other hand, if you are running a popular FTP mirror and routinely provide very large downloads to users, you may find this worthwhile.

If your server receives many large inbound requests, it might make sense to tune net.inet.tcp.recvspace. Candidates for large inbound network data might be mail servers that handle a lot of mail and deal with a lot of unusually large messages or web servers that deal mostly with very large file uploads. Again, this variable is only valuable if your traffic patterns are substantially dominated by specific kinds of well-characterized traffic. If you see a general mix of big and small, inbound and outbound traffic, then tuning these variables will have limited effect.

Novices often ask about tuning the number of mbufs available in the kernel because they see messages in their logfiles about running out of mbufs. In the modern network infrastructure of the kernel, no variable can be tuned in isolation; each affects another. However, FreeBSD does allow some influence over the mbufs by tuning the kern.ipc.nmbclusters variable. Before deciding that you need to tune this variable, though, you should run netstat -m to see if you are anywhere near your limits. Even on busy servers, the kernel generally picks good values for this variable.

In OpenBSD, the number of mbufs allocated is a function of the maxusers variable. This is a more coarse method of tuning than you might desire. If you are willing to reconfigure and recompile your kernel, you can set the NMBCLUSTERS variable in the kernel configuration file directly.