12.2 Some Common Virtual Domain Setups

Although qmail's virtual domain mechanism is extremely flexible, most of its applications fall into a few common models.

In each case, you must pick a local user or subaddress to assign the virtual domain to. It can be a real user in /etc/passwd, or if you use qmail's users mechanism (see Chapter 15), a qmail-only user. If you handle several virtual domains in the same way, all the domains can share a user, with delivery programs distinguishing among them by checking $HOST or $RECIPIENT. If you're only forwarding mail, you can handle virtual domains under ~alias; otherwise, it's a good idea to set up separate user IDs per virtual domain or per kind of virtual domain so that the programs you run can only smash that user's files when they break. Also, if you want to delegate the management of a virtual domain to someone else, make a separate Unix user account for the domains so the manager can log in as that user and change the domain's mail setup.

If you want people outside your network to be able to send mail to the virtual domain, you must list the domain in rcpthosts. If you're using a virtual domain as a service gateway for your own users to a fax server or the like, don't put it in rcpthosts.

Finally, each time you change the contents of virtualdomains or locals, you must tell qmail to reread them by sending a hang-up signal. Assuming you're running qmail under daemontools, use:

# svc -h /service/qmail

12.2.1 Mapping a Few Addresses with .qmail Files

In the simplest case, you can just create a .qmail file per address. Assume you have the domain myvirt.com, with addresses william, wilbur, and wilfred, which you want to forward to local addresses biff, buddy, and butch, respectively. This example doesn't do any special processing on the mail, so just set it up as a subuser of alias. Add this to virtualdomains:

myvirt.com:alias-myvirt

(Don't forget to signal qmail to reread the configuration.) Now you need to create only three .qmail files in ~alias and you're done:

.qmail-myvirt-william    &biff .qmail-myvirt-wilbur     &buddy .qmail-myvirt-wilfred    &butch

That's it. Mail to the three addresses is now forwarded to the three mailboxes.

In a realistic example, you'll probably want to define a few more standard addresses such as postmaster and abuse. Either you can create individual control files like .qmail-myvirt-abuse, or you can make a catchall file to collect mail to all other addresses in the domain, .qmail-myvirt-default. A catchall file catches mail to misspelled versions of the three explicit addresses as well as to other administrative addresses such as webmaster, hostmaster, and support. Unfortunately, the majority of mail to other addresses is likely to be spam. If you have a spam filter, aim the default file at a filtering program and deliver whatever survives the process, or do nothing about the process and bounce the mail. If you have no .qmail-myvirt-default in ~abuse, but there is a global catchall .qmail-default, the global catchall will handle all of the misaddressed myvirt mail, which you do not want. To bounce any misaddressed mail, put something like this in .qmail-myvirt-default:

| bouncesaying "Not a valid address at myvirt.com."

12.2.2 Mapping Many Addresses with fastforward

If you have more than a handful of addresses to forward, rather than creating dozens or hundreds of .qmail files, it's easier to put the forwarding instructions in one file and use fastforward to forward the mail. The table for fastforward is created by either newaliases (from /etc/aliases) or setforward (from any file you want). For virtual domains, setforward is more convenient.

If you have several virtual domains, either put the forwarding rules for each domain into a separate file or put the rules for several domains into a single file. Unless the rules for all the domains are identical or nearly so, it's easier to have a file per domain.

First, set up the virtualdomains line(s) for the domain in question. Each fastforward file needs a different user or subuser. Again, the forwarding can be via subaddresses of ~alias or else delegated to a different user. Here are some lines for three domains handled in a single file and a fourth domain handled separately, all in ~alias:

example.com:alias-example example.org:alias-example example.net:alias-example myvirt.com:alias-myvirt

Then create the .qmail files. In .qmail-example-default, put:

| fastforward -d example.cdb

And in .qmail-myvirt-default, put:

| fastforward -d myvirt.cdb

The --d flag tells fastforward to use $DEFAULT@$HOST as the address to forward rather than the normal $RECIPIENT. The difference is that with --d the user prefix is stripped off, so that even though mail to, say, [e-mail] is delivered to [e-mail], the address that fastforward uses is stripped back to [e-mail].

Put the forwarding instructions in files called example and myvirt, and set up Makefile to make the CDBs:

# makefile for two fastforward files example.cdb: example         /var/qmail/bin/setforward $@ $@.tmp < example myvirt.cdb: myvirt         /var/qmail/bin/setforward $@ $@.tmp < myvirt

Finally, make the files of forwarding instructions. As described in Chapter 4, the instructions are either addresses to forward or programs to run. In the single-domain file, each address is just a mailbox:

fred: phred

In the multidomain file, each address is just a mailbox if it applies to all of the domains, or it can include the domain if the address is handled differently in the various domains:

# [e-mail], [e-mail], and [e-mail] all same fred: [e-mail] # robert handled differently [e-mail]: bob [e-mail]: robbie [e-mail]: | bouncesaying "No such mailbox. Go away."

One thing that fastforward cannot do is deliver directly to a Maildir or mailbox. If a few of the addresses in a domain go to mailboxes, you can create .qmail files for those addresses, but handle everything else with fastforward. If, say, you want to put all the abuse mail in a mailbox, create .qmail-myvirt-abuse or .qmail-example-abuse containing:

./abuse.mbx

You can't distinguish among domains without running a command that tests the value of $HOST, so you can't distinguish domains and deliver to a mailbox in one step. If you want to deliver the three example abuse addresses to separate mailboxes, forward each of them to a different local address, then make .qmail files for each of those local addresses.

12.2.3 Per-User Subdomains

Although qmail makes it possible to give users an unlimited number of subaddresses using hyphens, some people dislike hyphenated addresses, either because they confuse their correspondents or because it's too easy to "untag" the address by removing the part after the hyphen. Using virtual domains, you can give each user a separate subdomain, so that, for example, if user ella has a subscription to the "mental" list and wanted to use a tagged address, she could use [e-mail], which qmail could internally handle as the subaddress ella-mental.

This trick turns out to be extremely easy to set up. First, put wildcard subdomain entries into virtualdomains (note the leading dot):

.myvirt.com:alias-sub

and into rcpthosts:

.myvirt.com

Then create .qmail-sub-default containing:

| forward "$HOST3-$[e-mail]"

You may want to treat a few administrative addresses specially, so that the system manager still gets all of the mail to postmaster and abuse. That's easily arranged (all on one line, although it's split here):

| case "$DEFAULT" in postmaster|abuse) forward $DEFAULT ;;     *) forward "$HOST3-$[e-mail]" ;; esac

Now mail to any subdomain of myvirt.com is handled as a virtual domain, and the .qmail file takes the subdomain ($HOST3, because it's the third component of the hostname from the right) sticks it in front of the existing mailbox name, and then forwards to that address.

This trick works well so long as all subdomains of a given domain are to be treated as mailbox addresses. If some subdomains are used for other purposes, such as hostnames or web server names, I find the results confusing, particularly if some of the hostnames appear in locals. Either use a virtual domain for mail that isn't used for anything else or, if the number of users is small, list the individual user subdomains in virtualdomains or rcpthosts, so mail to subdomains you don't use won't be accepted:

# in virtualdomains alan.myvirt.com:alias-sub barb.myvirt.com:alias-sub chad.myvirt.com:alias-sub debi.myvirt.com:alias-sub ella.myvirt.com:alias-sub fred.myvirt.com:alias-sub # in rcpthosts alan.myvirt.com barb.myvirt.com chad.myvirt.com debi.myvirt.com ella.myvirt.com fred.myvirt.com

12.2.4 Service Gateways

Virtual domains provide an elegant mechanism for gateways from SMTP email to other services. They're useful both for sending mail by other means such as uucp, and for nonmail services such as mail-to-usenet and mail-to-fax. The mailbox part of the address tells the gateway how to pass the message to the other service. For uucp, it is the username on the remote system, for a usenet gateway, the newsgroup, for a fax server, the phone number to fax to. It's also possible to encode other information in the address; in my local mail-to-news gateway, I encode hints about moderated groups and bounce handling of unforwardable messages, as described in the next section.

The general strategy for a service gateway is to create a virtual domain, then deliver all the mail for that domain to a program that performs the gateway function. Depending on how long the gateway program takes to run and how quickly you want messages passed along, you can either set up the gateway program to run each time a message arrives or collect all the messages in a Maildir and run the gateway program every once in a while to process all the messages in a batch.

12.2.4.1 Gateway design

When setting up gateways, take a few minutes to be sure that your setup will work well for your users. Most importantly, be sure that the gateway setup makes sense from the point of view of users, and not just for the convenience of the gateway administrator. One of the worst mistakes of sendmail's design is that the syntax of addresses depends on the scheme used to transport the mail to a remote system, a detail that users rarely care about. When setting up service gateways, be sure not to recreate the same mistake. If you're setting up a gateway to forward mail by uucp, don't create a local virtual domain called uucp with addresses like [e-mail] or faraway!fred@uucp, thereby forcing remote users to use awful hacks like [e-mail] or faraway![e-mail]. Instead, use a naming system that reflects the details that matter to the user. If faraway is a system that sends and receives mail, integrate its addresses into the rest of the mail addressing system, something like [e-mail]. This means, among other things, that if the remote system switches to a different mail system, user addresses don't have to change.

It's not hard to arrange your virtual domain setup so that the choice of the remote service is entirely handled by the virtualdomains file. Assume you have three uucp peers called faraway, distant, and pluto. Then you could set them up like this, and have the gateway program check $HOST or, more likely, $HOST3 to pick out just the hostname:

faraway.example.com:alias-uucp distant.example.com:alias-uucp pluto.example.com:alias-uucp

As discussed in the next section, you can easily handle all three with a single gateway.

12.2.4.2 Gateway addressing

A little planning when setting up your gateway can make it both easier for people to use and easier for you to implement. If the gateway is to another email system, the only sensible approach is to give the other system a domain name so mail to its users looks like mail to any other domain on the Net.

If the gateway is to another service, try to arrange for addresses to be easy to remember. For example, a fax gateway would take addresses like 13115552368@fax, but you might as well also accept 1-311-555-2368@fax and 311-555-2368@fax and, if all the users are in the same area code, 555-2368@fax. For a mail-to-news gateway, use addresses like alt.flame@news.

Sometimes it makes sense to embed some options into the gateway address. For example, in my mail-to-news gateway, I sometimes want the gateway to add an Approved: header to the posted article, and sometimes I want to avoid sending bounce messages about failed posts (newsgroups that log filtered spam, for example, which mostly has fake return addresses^[1]). The easiest way to encode flags into the address is with a prefix like approve-alt.flame@news and nobounce-local.spamtrap@news. You can also encode the flags into the domain, like [e-mail], but I've found prefixes to be easier to use.

^[1] If you create a local moderated newsgroup corresponding to a mailing list, set the moderator's address for that newsgroup to the list's submission address, and route incoming mail from the list to the mail-to-news gateway address with the approval option, you get a pretty good two-way gateway.

12.2.4.3 Per-message service gateways

Running the gateway program for each message is usually the easiest approach to implement. Create a .qmail-default file that runs the program, and either pass the value of $DEFAULT as a program argument or have the program pick it up from the environment.

To continue the uucp example, a simple gateway for a single system could be set up in virtualdomains like this:

faraway.example.com:alias-faraway

In ~alias/.qmail-faraway-default, it would be:

| uux -p "faraway!rmail" "$DEFAULT"

To generalize this for multiple uucp hosts, route all the uucp domains to alias-uucp, and create ~alias/.qmail-uucp-default containing:

| uux -p "$HOST3!rmail" "$DEFAULT"

Now the hostname, which precedes .example.com in the domain name, is picked out and handed to uucp. If you have a variety of mail gateways, you need to adjust only the lines in virtualdomains to control which domains are sent through which gateways.

For a gateway to a different kind of service, such as net news or fax, you must write your own gateway program in C or Perl to read through the message headers to pick up any lines the gateway needs, such as Subject or Date (not To, that comes from $DEFAULT), and pass the body to the program for the other service.

In the special but fairly common case that the gateway looks up an address in a database of some sort and remails the message, your gateway program can run qmail's forward program to do the remailing; there's no need to write your own if you don't want to.

If your gateway encodes options into the address (such as approve- in the previous section), you can easily handle them with an extra .qmail file, such as .qmail-approve-default, that calls the gateway program with whatever extra arguments it needs to implement the option.

12.2.4.4 Batched service gateways

Sometimes it's easier to run the gateway program periodically, either from cron or when some other event happens, such as when a remote system connects via PPP. In that case, you can have all of the mail for a virtual domain or group of virtual domains delivered into a Maildir, then have the gateway program take the messages out of the Maildir and do whatever it does with them, looking at the Delivered-To: line at the front of each message to tell what the message's recipient address is.

The serialmail package, discussed in Chapter 9, provides a general purpose framework for building batched gateways. The maildirserial program looks through a Maildir for files with a Delivered-To: line that matches a specified pattern (this lets several similar gateways share a Maildir), passes the names of matching files to a gateway program, reads delivery reports from the gateway, and optionally sends bounce reports for undeliverable messages.

When you run maildirserial, you give it the name of the Maildir, the address prefix, and the name of the gateway program to run. When it runs the gateway program, it sends to the gateway the names of files to process, and the gateway sends back delivery reports. The gateway program should process all the files that maildirserial gives it, but if it doesn't, maildirserial reruns the gateway until it either produces a delivery report for each file or makes no further progress.

The gateway program runs with the Maildir as its current directory, and pipes to and from maildirserial. The names of files to process arrive on the standard input, separated by nulls. It sends delivery reports to standard output. Each report consists of the filename and a null, as received from stdin, followed by a one-letter status code, an optional line of descriptive text for the logs or bounce message, and a newline. The status code is K if the message was delivered successfully, Z if the delivery failed temporarily and should be tried later, and D if the delivery failed permanently and the message should be bounced. If either the delivery succeeded or it failed permanently and a bounce message was sent, mailderserial will delete the file; otherwise the file stays so the delivery can be tried again.

As a concrete example, here's the framework of a Perl gateway program. It takes one command-line argument, the target address prefix. It reads null-separated filenames, opens the files, picks up the envelope sender and recipient addresses from the Return-Path: and Delivered-To: lines, does something with the file, and sends back a delivery report:

$prefix = shift or die "need prefix"; while(!eof STDIN) {     {   local $/ = "\0"; # read null separated file names         $fn = <STDIN>;         chop $fn;     }     open(MSG, $fn) or die "cannot open 'fn\n";     if(<MSG> =~ m{Return-Path: <(.*)>}) {         $sender = $1;     } else {         close MSG;         print "$fn\0Dno sender address\n";         next;     }     if(<MSG> =~ m{Delivered-To: $prefix(.*)}) {         $recip = $1;     } else {         close MSG;         print "$fn\0Dno recipient address\n";         next;     }     ### do something with the message here ###     close MSG;     print "$fn\0Kmessage delivered\n"; }

If this script were called ~/bin/gate, the local address is alias-myvirt, and the Maildir is called myvirtmail, then invoke it as:

setlock myvirtmail.lock \   maildirserial -b -t604800 myvirtmail alias-myvirt- \      ~/bin/gate alias-myvirt- 2>&1 | \ splogger serial

The call to setlock prevents two copies of maildirserial from running at once, and piping through splogger sends the results to syslog. Note that alias-myvirt- occurs twice, once for maildirserial and once for gate, and that it ends with a hyphen to prune off everything before the virtual domain mailbox. The --b flag tells maildirserial to bounce mail in case of a permanent failure, and the --t flag tells it to treat a temporary failure as permanent after a week. Other than for debugging, you should always include them, perhaps adjusting the time limit for temporary failures. Run maildirserial from cron if you want to push stuff through the gateway on a fixed schedule, or if the gateway depends on a network or dialup connection that's not always available, start it from the script that starts the connection, as described in Chapter 9.

When debugging your gateway program, rather than firing up maildirserial, mail a few test messages to your gateway that will land in your Maildir files, then use a text editor to create a file containing the names of those files, each followed by a null character (typed as Ctrl-V Ctrl-@ in vi, or Ctrl-Q Ctrl-@ in emacs). Then run your gateway with input redirected from that file. You can rerun it as often as you need to, because your gateway program doesn't delete its input files.

If your gateway program uses a TCP/IP connection to a remote system, place tcpclient between maildirserial and your gateway program to open the connection, like this:

setlock maildir.lock \  maildirserial -b -t 1209600 maildir prefix \   tcpclient -RHl0 host port \    gatewayprogram prefix

The gateway program reads and writes from and to the network on file descriptors 6 and 7, still using 0 (stdin) and 1 (stdout) for filenames and delivery reports. See Appendix A for a mail-to-news gateway using maildirserial and tcpclient. To debug programs running under maildirserial and tcpclient, I usually redirect the network connection to my tty so I can manually play the part of the remote server and step through the sequence of commands. To do that, use some shell redirections for the input file mentioned previously and the terminal:

gatewayprogram <listofinputfiles 6</dev/tty 7>/dev/tty

Internet services including SMTP and NNTP require a return character before the line feed at the end of each line, while input from /dev/tty just has a newline. To work around that, I write my scripts so they strip out the \r and work either way. Alternatively, if the server needs to send more input than you can easily type, put a sequence of input messages in a file, edit in the \r at the end of each line, and run the gateway program redirecting file descriptor 6 from that file.

The example code shown here handles the messages sequentially, one at a time in order, but maildirserial doesn't require that you do so. Your gateway can process the files it passes in any order, individually or all at once, so long as it sends back an appropriate status report for each file that it's processed. The status reports do not need to be sent in the same order that the filenames were received, and the program can receive as many filenames as it wants before sending back any reports. (maildirserial runs as two independent processes: one reading the Maildir and sending the filenames, and the other receiving the reports, deleting processed files, and sending bounces.)

If your gateway won't ever bounce back messages sent to invalid addresses or have a temporary gateway failure, you don't need to use maildirserial. Just run your program as needed (from cron or otherwise). It should read the Maildir's new subdirectory and process all the files it finds there, deleting each one as it's done. You should still use setlock or the equivalent so that you don't get multiple copies of the gateway running at once.

If you encode options into your gateway address or domain, your gateway program must decode the address from the Delivered-To: line to pick out the options. I write most of my gateway programs in Perl, so the decoding takes only a few lines of code.

12.2.5 Mapping Individual Addresses

A rarely used virtual domain option maps individual addresses in a virtual domain. It's primarily useful to short-circuit mail to local users who also have addresses in other places. Let's say you're at the two-person East Podunk office of your company, connected by a slow dialup line, with user addresses [e-mail] and [e-mail], but the company uses a standardized addressing scheme so to the outside world the addresses are [e-mail] and [e-mail]. If Fred sent a message to [e-mail], it has to go out over the dialup link and be forwarded back, probably a lot later. To avoid that, you can special-case the two addresses to be local:

[e-mail]:alias [e-mail]:alias

This routes these two addresses locally to alias-fred and alias-ethel, which you can handle with .qmail files, while leaving the rest of the example.com domain to be routed normally.

It's not very useful to override individual entries in local virtual domains, because you receive almost exactly the same effect by using .qmail files; that is, if the mail for fred is to be handled differently from everyone else's and everyone else's mail is handled by ~myvirt/.qmail-default, you can put Fred's special rules in ~myvirt/.qmail-fred.

12.2.6 POP Toasters

A POP toaster is a virtual domain in which all (or nearly all) of the mailboxes are routed to mailboxes that users can retrieve via POP and other remote access services. POP toasters are covered in detail in Chapter 13.