Preparing to Install qmail

---

Before you start installing qmail, you need to make some decisions about how you want to install it. Do you want to install a prebuilt package for your particular operating system? Or perhaps you want to install a source-code package like an RPM? Maybe you want to install qmail from the basic source-code tar file (tarball)? And where do you want to install it?

Even if you're an old hand at installing software, you should consider these issues carefully because qmail is unique in several ways that might affect your decisions.

Binary or Source Code?

Because of qmail's restrictive licensing regarding the distribution of prebuilt packages, qmail is usually installed from a source-code distribution.

If you're not familiar with the distinction between source code and binaries, imagine ordering a pizza delivered to your house. The "binary" version of the pizza arrives ready to eat. The "source-code" version of the pizza comes as a kit containing flour, yeast, cheese, sauce, toppings, and directions for cooking the pizza. Source-code installations are a little more work for you, but if you follow the directions carefully, the result is the same—or even better. And you'll know a lot more about your pizza and how it works.

Tarball or Operating System-Specific Package?

Some operating systems provide a mechanism for automating source-code installations. Returning to the pizza analogy, they make it possible to package the ingredients and directions in such a way that you can just push a button and have the pizza bake itself.

Sounds great, doesn't it?

In practice, this might not be such a good idea. Assembling these packages can be fairly difficult, and they might not work as intended. Like any software, they can have bugs. But even if these assemblies are bug free, the convenience they provide comes at a cost. You lose most of the advantages of the self-baked pizza, such as the ability to adjust the toppings to your personal preferences, the knowledge of how the pizza was made, and the knowledge of how it works.

If qmail was a pizza, the self-building approach might still be the way to go. But it's not: It's a fairly complex system that the installer/maintainer needs to understand pretty well to be able to keep it working smoothly. The self-installing qmail is easier to install than the user-installed version, but the user-installed version is easier to configure and troubleshoot. Configuring and installing from source code will also give you a greater understanding of how qmail works. You install qmail once on a system, but you will probably have several opportunities to reconfigure or debug it.

That's why installing qmail from scratch using the source-code tarball, not a Red Hat RPM or other self-installing bundle, is recommended. If you still want to install from RPM, it's covered in the "Installing from RPMs" section.

Choosing a Mailbox Format and Location

Messages received for final delivery are stored in a mailbox—a file or directory that contains messages delivered to a local address and owned by the user responsible for that address. The locations and formats of mailboxes vary depending upon the transfer, user, and delivery agents involved.

One of the most important decisions you'll make when installing qmail is the location and format of mailboxes. You basically have three choices:

• mbox mailboxes under /var/spool/mail or some other central spool directory

• mbox mailboxes under the user's home directory

• maildir mailboxes under the user's home directory

A fourth possibility, maildir mailboxes in a central spool directory, is not commonly used except in virtual user configurations, which is covered in Chapter 11, "Hosting Virtual Domains and Users." There are compatibility, security, convenience, and performance tradeoffs with each of these choices, so we'll look into them more deeply.

The Mbox Mailbox Format

Traditionally, Unix mailboxes have been stored in a centralized location, usually /var/spool/mail or some variation, in a simple, single file format called mbox. In an mbox mailbox, messages are prefixed with a line that looks like a From header field. The mbox man page describes the format in detail.

This is an example of a message in mbox format:

 From [e-mail] Fri Mar 09 12:00:39 2001-03-09 Return-Path: <[e-mail]> Delivered-To: [e-mail] Received: (qmail-queue invoked from smtpd); 9 Mar 2001 12:00:38 -0000 Received: from blossom.example.net (200@10.10.10.12)   by bubbles.example.com with SMTP; 09 Mar 2001 12:00:38 -0000 Received: (qmail 16464 invoked by uid 200); 9 Mar 2001 12:00:38 -0000 MIME-Version: 1.0  Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <[e-mail]> Date: Fri, 9 Mar 2001 08:00:37 -0500 (EST) From: Jessica <[e-mail]> To: [e-mail] Subject: Movie tonight Want to go see a movie tonight? >From, Jess 

The first line of the mbox, starting with the word From, is the mbox prefix line. If a message contains any other lines starting with From, the line must be escaped by prefixing it with a greater-than sign (>) as in the next-to-last line in the example.

Pros

mbox mailboxes are supported by almost all Unix MUAs—and are the only format supported by some of them. If you're migrating a system using mboxes to qmail, you might want to stick with the mbox format to avoid the need to convert existing mailboxes to the maildir format and to avoid compatibility issues with the MUAs your users are already using.

Cons

Because all messages are stored in a single file, mboxes require careful locking to prevent simultaneous updating by multiple user agents—which can result in mailbox corruption. The escaping of message lines starting with From is distracting to the user and interferes with message checksums and digital signatures. Also, there are several variants of the mbox format, as described in the mbox man page, and all MDAs and MUAs operating on a mailbox must use the same variant for reliable operation.

The Maildir Mailbox Format

qmail introduced the maildir mailbox format specifically to address the weaknesses of the mbox format. A maildir mailbox is, as the name suggests, a directory containing three subdirectories. Each message in a maildir is stored in a separate file in one of the three subdirectories, depending on the message's state. The three subdirectories are

• tmp for messages in the process of being delivered

• new for messages that have been delivered but not yet seen by the recipient

• cur for messages that have been seen by the recipient

Note 

Empty maildirs can be created using the maildirmake command, which must be run as the owner of the mailbox, not root. Most operating systems support a "skeleton" directory whose contents are copied to the home directories of new users. If you configure your qmail system to default to maildir delivery, it's a good idea to include a maildir directory in the skeleton directory. The traditional name for a user's primary maildir mailbox is Maildir.You can use any name you like, but Maildir is unlikely to result in confusion about the intended format of the mailbox.

Note 

One important difference between mboxes and maildirs is that qmail will not automatically create maildirs at the time of delivery. Be sure to create maildir mailboxes in advance or deliveries will be deferred.

Pros

No locking is required to limit simultaneous access by multiple user agents—even for maildirs accessed via Network File System (NFS). Because each message is stored in a separate file, there's no need for a message prefix or escaping lines starting with From.

Cons

The maildir format is relatively new, and some MUAs don't support it. If all mail-box access will be via Post Office Protocol (POP) or Internet Mail Access Protocol (IMAP), this isn't an issue because you can use POP and IMAP daemons that support maildirs. If you're migrating from mboxes to maildirs, you'll have to ensure that the MUAs you provide support maildirs, and you might want to convert the existing mboxes to maildirs.

Mailbox Location

Traditionally, each user's incoming mailbox is stored in a central spool directory such as /var/spool/mail, /usr/spool/mail, or /var/mail. This has been the source of numerous security problems because of holes in setuid() delivery agents and improper permissions on the mail spool directory, which let users take ownership of other users' mailboxes.

To avoid these problems, the default location for a user's mailbox under qmail is in their home directory. You can configure most MUAs through an environment variable or configuration file to accommodate this change. Table 2-1 shows how to specify the mailbox location.

Table 2-1: Specifying Mailbox Location Using an Environment Variable

MAILBOX FORMAT	MAILBOX LOCATION	VARIABLE	SETTING
mbox	/var/spool/mail/username	MAIL	/var/spool/mail/username
mbox	$HOME/Mailbox	MAIL	$HOME/Mailbox
maildir	$HOME/Maildir	MAILDIR	$HOME/Maildir
maildir	/var/spool/mail/username	MAILDIR	/var/spool/mail/username

Where to Put the Files?

Where should the binaries go? Where should the man pages go? The configuration files? The queue?

Every system administrator and operating system has different ideas about where the various pieces of a package belong. qmail is quite flexible about where these different pieces reside, but it wants all of them to be accessible from a single master directory. By default, everything will be installed under /var/qmail. You could change the master directory to /usr/local/qmail or whatever else you'd like, but by leaving it alone you make it easier for other people familiar with qmail to support your installation.

qmail makes a strong case for locating the master directory on the /var file system in the online qmail frequently asked questions (FAQ). The config, queue, alias, and users subdirectories fit the classic requirement for location in /var because their contents are system specific and should not be shared between machines. Because qmail compiles information about its various user IDs into the binaries, the bin subdirectory is also system specific. The remaining subdirectories—boot, doc, and man—are not system specific, but they're small, so they can be left in /var in most cases.

If you really want to make qmail conform to some preferred directory structure, the best way to accomplish that is to put the master directory in /var/qmail but relocate the subdirectories using symbolic links. Figure 2-1 shows how to do this.

Figure 2-1: Using symbolic links under /var/qmail

The following installation instructions include an example of such a reorganization.

Requirements for the Location of the Queue

Because of its high reliability, qmail imposes some requirements on the file system used to contain the queue.

Must Be Local

NFS and other network file systems don't behave exactly like traditional, local file systems. They're also much slower than local file systems.

Always use a file system for the queue on a disk physically attached to the qmail system.

Must Perform link() Calls Synchronously

qmail assumes that when it executes a link() call the metadata (directory and inode information) for the new link is safely written to disk. This is not true for all file systems and operating systems. File systems known to meet this requirement include

• BSD Fast File System (FFS) without "softupdates"

• FFS variants like Solaris or Tru64 UFS

File systems that don't meet this requirement include

• Linux Ext2

• BSD FFS with softupdates

• SGI XFS

• ReiserFS

There are a couple of Ext2-specific workarounds. First, the file system can be mounted with the "sync" option to cause all operations on the file system to be performed synchronously, or directly to disk. This can have a negative impact on performance, though, because it allows no caching, even when it could be done safely, and because it applies to all programs writing anywhere on the file system—not just qmail. Somewhat less drastic is to use the chattr command to set the S attribute on all of the queue subdirectories. That still prevents some safe caching, but at least it only affects qmail.

There are better workarounds that work on most file systems.

One is syncdir (http://www.untroubled.org/syncdir/), a library that provides versions of the standard library functions that modify metadata (open(), link(), unlink(), and rename()) with explicit calls to synchronize the metadata. Using this workaround requires installing syncdir and modifying the qmail Makefile to include -lsyncdir when loading qmail-queue. See Appendix B, "Related Packages," for detailed instructions.

Another workaround that may be right for you is a patch (http://www.jedi.claranet.fr/qmail-link-sync.patch) that adds explicit fsync() calls to synchronize the metadata. See Chapter 7, "Configuring qmail: Advanced Options," for more details about using patches.

Must Not Reside on a Disk Drive with Write Caching Enabled

This requirement is also somewhat tricky to determine. In general, Small Computer Systems Interface (SCSI) disks are less likely to enable write caching—unless the disk device driver requests it. Integrated Drive Electronics (IDE) drives are more likely to enable write caching by default.

Support Utilities

Like all systems of similar size and complexity, qmail requires—or at least works better with—the support of several system utilities. Traditionally, the network service "super server," inetd, is usually used to handle accepting connections on well-known ports and invoking the appropriate daemons associated with those ports. The logging service, Syslog, accepts messages from daemons, timestamps them, and writes them to a log file.

Unfortunately, these standard Unix utilities have some serious limitations. Luckily, Bernstein has designed replacements that address these problems. Although it's possible to use qmail with inetd and Syslog, it's not recommended even for "casual" installations.

Network Service

What's wrong with inetd? It handles many services on thousands of servers with-out any obvious problems, right? That's true, but not all of its problems are obvious. Most high-volume network services on Unix servers don't use inetd, including Sendmail and Apache. Here's why:

• inetd lacks a connection limit. There's no way to limit the number of active connections to a port. On a busy server, normal traffic levels can cause inetd to spawn more copies of a daemon than the system can handle. Denial-of-service attacks are easily perpetrated against services managed by inetd.

• inetd implements a connection-rate limit. If connections come "too fast" to a particular port, inetd reports that the service "may be looping" and disables it for some period of time. This is disastrous on a busy server—shutting it down at its busiest time. Some versions of inetd allow the maximum connection rate to be configured, but some don't. It also allows attackers to temporarily turn off a service simply by connecting to it until inetd disables it!

Bernstein's inetd replacement, Unix Client-Server Program Interface for TCP (ucspi-tcp) includes tcpserver, a simple utility that listens to a particular port—like 25 for SMTP—and invokes a daemon—like qmail-smtpd—for each connection to the port. (See Appendix B, "Related Packages," for more information on ucspi-tcp.) tcpserver implements a maximum concurrent connection limit as well as host-based access control similar to that provided by Wietse Venema's tcp_wrappers utility (ftp://ftp.porcupine.org/pub/security/index.html).

Logging

What's wrong with Syslog? It handles logging for many services on thousands of servers without any obvious problems, right? Again, that's true, and again the problems with Syslog are not so obvious that one runs into them daily. But they are serious problems, nonetheless:

• Syslog is inefficient. On a server with a busy service logging via Syslog, it's not unusual for the syslogd process to consume more processor cycles than the daemons providing the service. Some Syslog implementations include an option to write log files asynchronously, which does improve performance—at the cost of reliability.

• Syslog is unreliable. Syslog will completely fill the disk partition holding a log file if enough messages are logged. It contains no mechanism to limit the size of the logs. If it can't write to a log file, it'll simply throw away new log messages! Logging via the network, it uses User Datagram Protocol (UDP), not Transmission Control Protocol (TCP), so network problems can cause messages to disappear without a trace.

• Syslog is insecure. There have been several vulnerabilities discovered in syslogd, including those resulting in remote access to root. syslogd implements a network-logging service, but provides no access-control mechanism: Any system on the Internet can send a log message to your Syslog daemon.

Bernstein's Syslog replacement is multilog, from his daemontools package (see Appendix B, "Related Packages"). multilog automatically limits the log's size by rotating log files. It keeps a configurable number of old logs after rotating them, letting the system administrator configure exactly how much disk space will be devoted to a given service. It optionally timestamps log entries with up to nanosecond precision—compared to Syslog's one-second resolution. multilog also implements pattern matching for specifying entries to be included or excluded.

Process Control

Bernstein's daemontools package is a set of utilities for managing services. In addition to multilog—discussed in the previous section—it includes supervise, a service monitor, and various other tools for controlling processes monitored by supervise:

• svc is a service control program. svc allows the administrator to reliably and easily start, stop, or signal a daemon. Rather than using a process ID stored in a file—which might be incorrect—or requiring the administrator to parse the process ID from a ps listing—an error-prone process—svc passes the request through supervise.

• svstat displays the status of a service.

• svscan starts and monitors a collection of services. For each directory in the target service directory, svscan starts a supervise process to manage each service and, optionally, another supervise process to manage logging the service's output.

• tai64nlocal converts a multilog timestamp to a human-readable time/date format in the local time zone.

• setuidgid runs the specified command with the login user ID (UID) and group ID (GID) of the specified account.

• softlimit runs the specified command with specified resource limits.

Note 

The qmail installation presented in this chapter uses daemontools to start and control the qmail processes. A shell script interface is provided that implements a generic control interface similar to apachectl for the Apache Web server or ctlinnd for the InterNetNews (INN) server. For more information about apachectl, see the man page (http://httpd.apache.org/docs/programs/apachectl.html). For more information about ctlinnd, see the man page (http://www.mibsoftware.com/userkt/inn/doc/ctlinnd.8.htm).

---