Hack 69 Tune FreeBSD for Different Applications
|
Know how to tune and what to tune on your FreeBSD system As an administrator, you want to tune your server systems so they work at peak efficiency. How do you know what to tune? The answer depends heavily upon the system's function. Will the system perform a lot of small network transactions? Will it perform a small number of large transactions? How will disk operations factor in? How you answer these and other questions determines what you need to do to improve the performance of your systems. This hack starts with general optimizations and then looks at function-specific tunables. 7.2.1 Optimizing Software CompilingA good place to start is with software compiling, as you want to compile software and updates as efficiently as possible. Whenever you compile, your compiler makes assumptions about your hardware in order to create binaries. If you have an x86-compliant CPU, for example, your compiler will create binaries that can run on any CPU from a 386 onward. While this allows portability, it won't take advantage of any new abilities of your CPU, such as the extended MMX, SSE, SSE2, or 3DNow! instruction sets. This is also why using precompiled binaries on your system is a surefire way to reduce your overall performance. To ensure that software will be compiled efficiently, update your compiler flags in /etc/make.conf . This file does not exist on new systems, but you can copy it from /usr/share/examples/etc/defaults/make.conf. Start by editing the CPUTYPE= line to reflect your CPU type; you'll find supported types listed as comments just before this line. While this will take advantage of your CPU's features, the disadvantage is that your compiled binaries may not run on different CPU types. However, if all of your systems run the same CPU platform, any optimizations you make to shared binaries will affect all of your systems equally well. Next, change the CFLAGS line to CFLAGS= -O2 -pipe -funroll-loops. The -pipe option can significantly decrease the amount of time it takes to compile software, by using pipes to communicate between compiler processes instead of temporary files, but at the expense of using slightly more memory. The -funroll-loops saves one CPU register that would otherwise be tied up in tracking the iteration of the loop, but at the expense of making a slightly larger binary.
7.2.2 Kernel OptimizationsIn your kernel configuration, add the following line after the machine i386 line: makeoptions COPTFLAGS="-O2 -pipe -funroll-loops -ffast-math" This is similar to the CLAGS option in /etc/make.conf, except that it optimizes kernel compilation.
You can also add this line: TOP_TABLE_SIZE=number where number is a prime number that is at least twice the number of lines in /etc/passwd. This statement sets the size of the hash that top uses. Set the following option if you have an AMD K5/K6/K6-2 or Cyrix 6x86 chip. It enables cache write allocation for the L1 cache, which is disabled by default for these chips. options CPU_WT_ALLOC This option will disable NFS server code, so include it when you know that you will not be acting as an NFS server: options NFS_NOSERVER Another way of saving kernel memory is to define the maximum number of swap devices, as shown in the next example. Your kernel needs to allocate a fixed amount of bitmapped memory so that it can interleave swap devices. I set the number to 1 on my workstation and 2 on my servers. If I need to add more to a server, I can easily create another partition. options NSWAPDEV=number If you plan on compiling all your requisites into the kernel (NIC driver, IPF/IPFW, etc.) and won't be loading any of these options as modules, you can include this line to skip module compiling. This saves significantly on the time taken to compile a kernel (sometimes reducing it by two-thirds). makeoptions MODULES_OVERRIDE="" By default, all kernel options are compiled as modules. This allows you to use kldload to load a module even though it isn't specified in your kernel configuration file. The advantage of MODULES_OVERRIDE is the decrease in kernel compilation time. The disadvantage is that you'll need to recompile your kernel if you ever need to add additional functionality, since you will have lost the ability to load the kernel module separately. 7.2.3 Optimizing Network PerformanceMost modern network cards and switches support the ability to auto-negotiate the communication speed. While this reduces administration, it comes at the cost of network throughput. If your switch, server, or workstation is set to use auto-negotiation, it will stop transferring network traffic every few moments to renegotiate its speed. If your network driver supports it, you can set network speed with ifconfig at runtime or in /etc/rc.conf at boot time. Here is an example: % grep fxp0 /etc/rc.conf ifconfig_fxp0="inet x.x.x.x netmask x.x.x.x media 100BaseTX mediaopt full-duplex"
Next, you can enable DEVICE_POLLING in your kernel, which changes the method by which data travels from your network card to the kernel. Without this setting, frequent interrupt calls may never free the kernel. This is known as livelock and can leave your machine unresponsive. Those of us unfortunate enough to be on the wrong side of certain denial-of-service attacks know about this. The DEVICE_POLLING option causes the kernel to poll the network card at certain predefined times, during idle loops, or on clock interrupts. This allows the kernel to decide when it is most efficient to poll a device for updates and for how long, and ultimately results in a significant increase in performance. To take advantage of DEVICE_POLLING, you need to compile two options into your kernel: options DEVICE_POLLING and options HZ=1000. The latter option slows the clock interrupts to 1,000 times per second, which prevents the kernel from polling too often. Once you've recompiled your kernel, you'll still need to enable the feature. Add this line to /etc/sysctl.conf: kern.polling.enable=1 The DEVICE_POLLING option does not work with SMP-enabled kernels by default. If you are compiling an SMP kernel with DEVICE_POLLING, first remove the following lines from /usr/src/sys/kern/kern_poll.c: #ifdef SMP #include "opt_lint.h" #ifndef COMPILING_LINT #error DEVICE_POLLING is not compatible with SMP #endif #endif 7.2.4 Optimizing Mail ServersMail servers typically have a very large number of network connections, during which they transfer a small amount of data for a short period of time before closing the connection. In this case, it is useful to have a large number of small network buffers. Network connections have two buffers, one for sending and one for receiving. The size of the buffer dictates how quickly data will funnel through the network and, in the event of a network delay, how much data can back up the server for that connection before there is a problem. Having a network buffer that is too small will cause a data backlog as the CPU waits for the network to clear, which causes greater CPU overhead. Having a network buffer that is too large wastes memory by using the buffer inefficiently. Finding a balance is the key to tuning. I find that multiplying the number of established connections by 32 leaves me with room to breathe in the event that I see an abnormally high surge of traffic. I've come to this number over time through trial and error. So, if you expect to have a peak of 128 servers sending you mail, having 8,192 network buffer clusters would be good (128 2 per connection 32). Also, remember that connections can take up to two full minutes or more to close completely. If you expect more than 128 emails in any given two-minute period, increase the number accordingly. Another important value to control is the maximum number of sockets. Start with the same number of sockets as there are network buffers, and then tune as appropriate. You can find out how many network buffer clusters are in use with the command netstat -m. You can specify the values you want in /boot/loader.conf. For example: kern.ipc.nmbclusters=8192 kern.ipc.maxsockets=8192 As with any performance tuning, monitor your system after making changes. Did you go overboard or underestimate what you would need? Always check and adjust accordingly. 7.2.5 Optimizing File ServersFile servers generally have longer-lived and less frequent network connections than those on mail servers. They usually transfer larger files. To determine the optimal number of network buffer clusters, consider how many clients you have. Multiplying the number of network buffers by two is good practice, though some admins prefer to multiply by four to accommodate multiple file transfers. If you have 128 clients connecting to the file server, set the number of network buffer clusters to 1,024 (128 2 per connection 4). 7.2.6 Optimizing Web ServersIf you have more than one element on your web page (for example, multiple images or frames), expect web browsers to make multiple connections to your web server. It's common to see four connections per page served. Also count any database or network connections made in server-side scripting. Web servers go through periods of highs and lows. While you might serve 100 pages per minute on average, at your low you might serve 10 pages per minute and at peak over 1,000 pages per minute. At a peak of 1,000 pages per minute, your clusters and sockets should be around 16,384 (1,000 pages 2 per connection 4 connections 2 for growth). 7.2.7 See Also
|
