Receiving Messages

---

Using qmail's .qmail files, users can direct incoming messages to mailboxes in two different formats, forward them on to other addresses, or process them automatically using scripts and programs such as vacation reminders, filters, and other message delivery agents (MDAs). Using .qmail files, users can manage their private address space: creating and deleting addresses. To make this easier, .qmail can "wildcard" match addresses: One .qmail can process all addresses matching a specified prefix.

For compatibility with Sendmail, Bernstein's dot-forward package implements message disposition via .forward files. Because Sendmail's .forward wasn't designed for user-managed address spaces, dot-forward is less useful than the dot-qmail mechanism. It's mostly used when migrating from Sendmail to qmail. Because dot-forward isn't part of qmail, we'll cover installation as well as usage.

Dot-qmail Files

The delivery of a user's mail is controlled by one or more .qmail (also known as dot-qmail) files. Dot-qmail files reside in the user's home directory and have names beginning with .qmail. The dot-qmail man page describes using .qmail files.

Dot-qmail files contain a list of delivery instructions, one instruction per line (see Table 4-3). Each line's first character determines what kind of delivery is involved.

Table 4-3: Dot-qmail Delivery Types

CHARACTER	DELIVERY TYPE	VALUE
#	None (comment)	Ignored
|	Program	Command to be run by shell
/ or .	mbox (if last char isn't a /)	Path name of mbox (including the leading / or .)
/ or .	maildir (if last char is a /)	Path name of maildir (including the leading / or .)
&	Forward	Address to which message will be forwarded
Letter or number	Forward	Address to which message will be forwarded (including the leading letter or number)

Path names starting with dot (.) are relative to the user's home directory.

Program Delivery

When a program delivery instruction is encountered, qmail starts a shell (/bin/sh) to execute the command and feeds the command a copy of the incoming message on standard input. The qmail-command man page documents the details of this process.

Program delivery is powerful and can be used to implement a wide range of functionality such as message filtering, automatically responding to messages, and delivery via third-party delivery agents such as Procmail.

For example:

 |preline /usr/ucb/vacation maryjane 

This causes qmail to start preline, pass it /usr/ucb/vacation and maryjane as arguments, and provide a copy of the message on standard input.

Mbox Delivery

mbox is the traditional Unix mailbox format, in which multiple messages are stored in a single file and messages are headed with a From line. This line looks like a header field, but it isn't: It's just something the delivery agent adds so mail readers can tell where each message begins.

For example:

 ./Mailbox 

This causes messages to be appended to $HOME/Mailbox, with a From line prepended. A simple mbox mailbox with a single message looks like this:

 From [e-mail] Thu May 13 18:34:50 2001 Return-Path: <[e-mail]> Delivered-To: [e-mail] Received: (qmail 1287205 invoked from network); 13 May 2001 18:34:49 -0000 Date: 13 May 2001 18:34:21 -0000 Message-ID: <[e-mail]> From: [e-mail] To: [e-mail] Subject: hey What's up? 

The first line was added at delivery by qmail.

Maildir Delivery

Bernstein created the maildir mailbox format to address the shortcomings of the mbox format. A maildir mailbox is a directory containing three subdirectories: new, cur, and tmp. Each message in a maildir mailbox is contained in a separate file in one of the subdirectories, depending upon its status: new is for unread messages, cur is for messages that have been seen, and tmp is for messages in the process of being delivered. The maildir man page describes the format of a maildir in detail.

One of the benefits of the maildir format is that, even though it doesn't use locking to prevent simultaneous updates from different delivery agents, it's reliable. This means maildir mailboxes can safely reside on Network File System (NFS)–mounted file systems.

For example:

 ./Maildir/ 

This causes messages to be saved in $HOME/Maildir, a maildir-format mailbox.

Note 

qmail-local can deliver mail to maildir mailboxes, but it can't create them. Maildir mailboxes should be created with the maildirmake program that comes with qmail. For example: maildirmake ~/Maildir.

Forward Delivery

Forward deliveries cause the message to be re-sent to the specified address. Addresses specified in .qmail files can't contain comment fields or extra spaces.

These are wrong:

 &<[e-mail]> & [e-mail] &Don User <[e-mail]> 

These are correct:

 &[e-mail] [e-mail] &don 

The first two cause [e-mail] to receive a copy of the message. The last sends a copy to the local user don.

Multiple Deliveries

In some cases, you'll want messages delivered more than once. For example, you might want to file a copy in a local mailbox as well as forward to another address:

 ./Maildir/ dave@mash 

That will save a copy of each message in $HOME/Maildir and forward another copy to dave@mash.

Note 

Unlike Sendmail, qmail won't recognize an instruction to forward a copy to yourself as a request to deliver a copy to your main mailbox. If you want to keep a copy, you'll have to tell qmail where to put it. If you do forward a copy to your-self, qmail will detect the loop and bounce the message.

Error Handling

If any delivery in a dot-qmail file fails, qmail-local stops processing the file immediately and returns an error. Entries in dot-qmail files are executed in order with one exception: All forward deliveries are saved for last. This means that if any delivery fails, none of the forward deliveries will be attempted regardless of the order of entries in the file. If the dot-qmail file contains:

 dave@mash ./Maildir/ 

And the maildir delivery fails—because Maildir doesn't exist, has the wrong format, has the wrong owner or mode, or any other reason—qmail-local will not forward a copy to dave@mash.

Tip 

You can "uncouple" the deliveries in a dot-qmail file by making them all forward deliveries. Instead of mixing mail-box and program deliveries with forward deliveries, have mailbox deliveries forward to another dot-qmail file first. For the previous example, you could change the ./Maildir/ entry to username-maildir and create a .qmail-maildir file containing ./Maildir/. Now, even if the Maildir delivery fails, a copy will be sent to dave@mash.

Hard and Soft Errors

If the program returns an exit code of 0 (zero), the delivery is considered successful and the remaining deliveries, if any, are processed normally. If it returns 99, the delivery is still considered successful, but remaining deliveries are skipped.

Any other exit code is considered a failure. If the exit code is 111, qmail considers the failure "soft" (temporary) and will retry the delivery periodically. If the exit code is 100, qmail considers the failure "hard" (permanent) and immediately generates a bounce message to the envelope sender. The output of the program, if any, will be included in the bounce message.

Most other exit codes are considered soft. Currently 64, 65, 70, 76, 77, 78, and 112 are considered hard, but this is subject to change. If you have the choice, use either 100 or 111 to be safe.

For example, user donna wants to bounce all mail sent to donna-junk. In $HOME/.qmail-junk she can put this:

 |exit 100 

which will immediately generate a bounce message back to the sender. If she wants to include an explanation for the bounce, she can include an echo command:

 |echo "This address is disabled." && exit 100 

Or, using the bouncesaying utility described below:

 |bouncesaying "This address is disabled." 

Conditional Delivery

In the case of program delivery, the exit status returned by the program determines how qmail-local will process remaining dot-qmail deliveries. If the program returns an exit code of 99, qmail-local will ignore all of the following deliveries, but it will still honor preceding forward deliveries.

This behavior can be used to implement conditional delivery—if X is true, then deliver the message, else bounce the message.

For example, say user donna wants the address donna-website to exist and deliver to the maildir $HOME/Mail/website but only if the From header field contains website.com. She could create $HOME/.qmail-website with these contents:

 |(grep "^From:" |fgrep -i "website.com" >/dev/null) || exit 100 ./Mail/website/ 

The first line extracts the From field from the message and searches it for website.com. The second line delivers the message to the website maildir—but only if the first line is successful.

Tip 

Bernstein's mess822 package, available from http://cr.yp.to/mess822.html, provides a utility called 822field that reliably extracts a given header field from a message. The grep command in the previous example would also match a From: website.com line in the body of the message. Also, remember that header fields are easily forged.

Extension Addresses

qmail supports user-controlled extension addresses. In addition to the base address, [e-mail], users can receive mail at [e-mail]. For the remainder of this section, we'll leave off the @hostname.domain part because we're talking about local deliveries.

The delivery instructions for username-extension are stored in ~username/.qmail-extension, the file .qmail-extension in user username's home directory.

For example, [e-mail] is controlled by ~dave/.qmail-tqh on host sparge.

Extensions can have multiple fields. For example, dave-list-old97s would be controlled by ~dave/.qmail-list-old97s. In this example, dave-list-old97s is subscribed to the old97s mailing list, and ~dave/.qmail-list-old97s files the list messages in a separate mailbox.

.qmail files can be wildcarded using -default. So dave-list-old97s could also be handled by ~dave/.qmail-list-default. This would allow one catchall .qmail file to handle all dave-list-whatever addresses. Note that dave-list wouldn't be handled by ~dave/.qmail-list-default because it doesn't match the hyphen (-) after list. It would be handled by ~dave/.qmail-list or ~dave/.qmail-default.

qmail uses the closest match it finds. When a message comes in addressed to dave-list-old97s, it'll use the first one of the following that it finds:

 .qmail-list-old97s .qmail-list-default .qmail-default 

If no matching .qmail file is found, the delivery defaults to the special user alias, where qmail looks for matching system aliases.

qmail-command Environment Variables

qmail-local uses several environment variables to provide useful information to commands run through dot-qmail files (see Table 4-4).

Table 4-4: qmail-command Environment Variables

VARIABLE	CONTENTS
SENDER	Envelope sender address
NEWSENDER	Forwarding envelope sender address, as described in dot-qmail
RECIPIENT	Envelope recipient address
USER	Local user's username
HOME	Local user's home directory
HOST	Domain part of the recipient address
LOCAL	Local part of the recipient address
EXT	Address extension
HOST2	Portion of HOST preceding the last dot
HOST3	Portion of HOST preceding the second-to-last dot
HOST4	Portion of HOST preceding the third-to-last dot
EXT2	Portion of EXT following the first dash
EXT3	Portion of EXT following the second dash
EXT4	Portion 0f EXT following the third dash
DEFAULT	Portion of LOCAL matched by -default in a dot-qmail file
DTLINE	Delivered-To line, including newline
RPLINE	Return-Path lines, including newline
UFLINE	UUCP-style From line that qmail-local adds to mbox-format deliveries

Caution 

Because these environment variables are set from the contents of messages supplied by potentially malicious users, they may contain characters with special meaning to the shell used to run the command. Users should take care to quote them when referencing them.

Examples

A message is sent to [e-mail], which is handled by .qmail-ext1-ext2-ext3-ext4-ext5. The message was sent by [e-mail]. Table 4-5 shows the results.

Table 4-5: Example qmail-command Environment Variable Settings

VARIABLE	SETTING
SENDER	[e-mail]
NEWSENDER	[e-mail]
RECIPIENT	[e-mail]
USER	dave
HOME	/home/dave
HOST	mash.example.com
LOCAL	dave-ext1-ext2-ext3-ext4-ext5
EXT	ext1-ext2-ext3-ext4-ext5
HOST2	mash.example
HOST3	mash
HOST4	mash
EXT2	ext2-ext3-ext4-ext5
EXT3	ext3-ext4-ext5
EXT4	ext4-ext5
DEFAULT	unset
DTLINE	Delivered-To: [e-mail]\n
RPLINE	Return-Path: Return-Path: <[e-mail]>\n
UFLINE	From [e-mail] Sun May 06 16:40:30 2001\n

If the same message was sent to the same address but was handled by .qmail-ext1-ext2-ext3-default, the environment would be identical except that DEFAULT would contain ext4-ext5.

Filtering Mail

In the early days of the Internet, most users had all incoming mail delivered to a single mailbox. These days, many users are finding it desirable, or even necessary, to split their incoming mail into multiple mailboxes, depending on where it came from, such as a particular user or site, or where it was sent, such as to a mailing list or extension address.

Before qmail, MTA support for user-managed address spaces was uncommon, so most of this splitting was done using filtering MDAs like Procmail or Maildrop. These MDAs allow the user to perform pattern matches against incoming messages and to direct them to more specific mailboxes.

Unfortunately, filtering is complicated, fragile, and expensive. It's complicated because the powerful filtering requires powerful tools, and powerful tools are complex. It's fragile because the Internet is dynamic. Users change Internet service providers (ISPs), and their addresses change—breaking filtering rules based on their address. ISPs change software and configurations, changing the contents of message sent by their users, potentially breaking filtering rules. Filtering is expensive because each delivery requires starting up the filtering MDA, which must then parse the filter rules, parse the message, and deliver the message accordingly.

Luckily, qmail's user-managed address space makes filtering largely obsolete. Instead of giving everyone a single address that dumps into a single in-box, users can create new addresses as needed for new purposes and efficiently and reliably direct messages to those addresses to different mailboxes.

Extension Addresses

Using extension address to direct incoming mail is easy. Before giving out your e-mail address to friends and family, subscribing it to mailing lists, and registering it on Web sites, ask yourself if you really want mail from this sender to land in your main in-box. Web site registrations and high-volume mailing lists are prime candidates for dedicated extension addresses.

Once you decide to give out an extension address, you have to do two things:

1. Choose a unique, self-identifying extension.

2. Set up a dot-qmail file to handle the new extension.

If you're subscribing to a mailing list, a good extension address might be username-list-listname. If you're filling out a Web site registration that requires an e-mail address, you might use username-web-website. In either case you could leave out the -list- or -web- part, but there are a couple reasons to use them. First, they help document the intended purpose of the address and prevent conflicts in the event that a list and Web site have the same name. Second, they allow you to set up -default dot-qmail files that catch entire classes of extension addresses.

If you use username-listname or username-website, you'll have to create .qmail-listname or .qmail-website before the associated address will work. That means you can't spontaneously create a new address while you're away from computer—without there being some chance of bouncing mail to that address before you create the dot-qmail file.

If you instead create .qmail-list-default and .qmail-web-default and direct them to generic, temporary mailboxes like $HOME/Mail/list/default and $HOME/Mail/web/default, you can give out new -list and -web addresses "on the fly" and create specific .qmail-list-listname or .qmail-web-website files at your convenience. Any mail sent to the new addresses before then will simply go to the default mailbox.

Subscribing Extension Addresses to Mailing Lists

How you subscribe an extension address to a mailing list depends upon how the list is managed.

• If you subscribe via a Web form, simply enter the extension address in the form (see Figure 4-1).

• If you subscribe via e-mail, the subscribe command might allow you to specify the address. For example, use subscribe listname user-list-listname for Majordomo or listname-subscribe-username- list-listname=example.com@listserver for ezmlm.

• If you subscribe via e-mail but the subscribe command doesn't allow you to specify the address, format the request so that the extension address appears to be the sender—either by specifying the From header with your MUA or by using qmail-inject environment variables.

Figure 4-1: Subscribing via a Web form

Procmail

Procmail is a popular MDA. The function of an MDA is to accept a message from the MTA for a specific user or mailbox and deliver the message according to the user's desires. Procmail can be used to filter messages based upon the content of various header fields or the body of the message. For example, messages from a particular person can be directed to a mailbox for just that person.

There are a couple tricks to running Procmail with qmail. First, procmail is usually built to deliver to an mbox mailbox in /var/spool/mail. You can rebuild procmail to default to $HOME or you can instruct users not to rely on procmail to default the location of the mbox. Unless you patch it for $HOME delivery, procmail will still use /var/spool/mail for its temporary files.

Another problem is that qmail-local and procmail don't have a common interpretation of exit codes. procmail uses the standard Unix exit codes: Zero means success, nonzero means failure, and the cause of the failure is indicated by /usr/include/sys/errno.h. qmail-local uses certain non-zero codes to indicate permanent errors and the rest are considered temporary. A small shell script wrapper can be used to translate the exit codes for qmail-local (see Listing 4-4).

Listing 4-4: qmail-procmail

 #!/bin/sh # Copyright (c) 1998-2001 Software in the Public Interest # <http://www.debian.org/> # Written by Philip Hands. Distributed under the GNU GPL # Modified slightly by Dave Sill preline /usr/bin/procmail && exit 0 # check if procmail returned EX_TEMPFAIL (75) [ $? = 75 ] && exit 111 # otherwise return a permanent error exit 100 

Older versions of Procmail (prior to 3.14) don't deliver directly to maildir-format mailboxes. Your best bet is to upgrade to the current version of Procmail. Another approach is Safecat, a program that writes a message on standard input to a specified maildir. Users can write Procmail recipes (delivery instructions) that use safecat to file the message in a maildir.

Finally, procmail expects the messages it receives to be in mbox format. Normal qmail program deliveries include only the actual mail message, not including a From line. The preline command (see "User Utilities") can be used to format the message as procmail expects.

For example, let's say a user wants his mail to be processed by procmail. His system administrator has built procmail to deliver to $HOME by default and has installed the wrapper above in /usr/local/bin/qmail-procmail. His .qmail file should look like this:

 |/usr/local/bin/qmail-procmail 

How procmail filters and delivers mail is determined by the .procmailrc file. See the procmailrc man page for a description of the format and the procmailex man page for examples.

Of course, Procmail must be installed on your system before you can use it. See the Procmail home page (http://www.procmail.org/) for more information on installation.

Maildrop

Maildrop is a filtering MDA with capabilities similar to Procmail. It was designed to work with qmail, so preline and exit code wrapping is unnecessary.

Invoking maildrop from .qmail is straightforward:

 |maildrop 

How maildrop filters and delivers mail is determined by the .mailfilter file. See the maildropfilter man page for a description of the format and the maildropex man page for examples.

Of course, Maildrop must be installed on your system before you can use it. See the Maildrop home page (http://www.flounder.net/~mrsam/maildrop/) for more information on installation.

---