9.5 SMTP-Time Filtering Tools

Once qmail-smtpd has started, filters can use the message envelope and data to trigger more filter rules. Some of the filters require patching the filter code into qmail-queue, while others can use the QMAILQUEUE patch to run the filters on the incoming message before queueing it for delivery.

9.5.1 Filtering in the SMTP Daemon

The three most useful checks in the daemon itself are MAIL FROM rejection, RCPT TO rejection, and Windows EXE virus rejection.

The standard qmail control file badmailfrom lists addresses and domains to reject as MAIL FROM arguments. The addresses are listed literally, domains preceded by @, so an address [e-mail] is rejected if either [e-mail] or @example.com appears. The rejection actually happens at subsequent RCPT commands because it's clearer to some SMTP clients that the mail can't be delivered.

I wrote a "badrcptto" patch, available at qmail.org, that lets you list recipient addresses to reject by putting them in badrcptto or morebadcptto, which is compiled into morebadrcptto.cdb by the new program qmail-newbrt. It only lists addresses; the way to reject recipient domains is to not put them in rcpthosts. The rejections happen after the DATA command to deter dictionary validation attacks. (Typical dictionary attacks start by trying a garbage address or two, in order to see whether the recipient MTA rejects them, and if they're not rejected, the attacker goes away.) The main point of badrcptto is one of efficiency. My system has a lot of addresses that get nothing but spam, and it's much faster to reject mail to those addresses at SMTP time than at delivery time. Also, if the message has multiple RCPT TO recipient addresses, it's rejected and not delivered to any of them if any of the addresses appear in badrcptto, on the theory that one can presume that any message sent to a known-to-be-bad address is spam even if it's also sent to a valid address. Another minor point is that rejecting at SMTP time sends the rejection to the actual sending host, rather than to the innocent return address, in the usual case that the return address is a fake.

There's a "goodrcptto" version of my patch floating around that flips the sense of the test and accepts mail only to listed addresses. I don't suggest you use it, because it breaks mail sent to subaddresses and -default addresses, some of qmail's most useful features.

The third daemon check deals with viruses. I observed in 2002 that all current viruses are Windows .exe files, and it's rare for anyone to send mail with an individually attached .exe files that's not a virus. Russ Nelson wrote a simple and extremely effective anti-virus patch, available at www.qmail.org, that recognizes the fixed code pattern present at the beginning of each .exe file. I suggest you use it, and tell your users who just have to mail around .exe files to put them in ZIP files before sending.

There are some other filtering patches for the SMTP daemon, none of which I recommend. One fairly popular one does a DNS lookup on the domain of each MAIL FROM address and rejects any that don't resolve. Several years back, a lot of spam used nonexistent fake addresses, but once the DNS checks became popular, spammers started forging genuine domains to defeat the DNS check. Nowadays, the DNS check slows mail delivery, because it can require a round-trip DNS lookup to a faraway DNS server, but stops almost no spam.

9.5.2 Separate Filters Called from the SMTP Daemon

Once qmail-smtpd has collected the incoming message, it normally runs qmail-queue to queue the message for delivery. If you've installed the QMAILQUEUE patch recommended in Chapter 3, it will instead run whatever program is named by the QMAILQUEUE environment variable. In practice you almost always run a shell script that calls the various filtering programs. In the simplest case, run incoming mail through a filter and then queue it. For example, create /var/qmail/bin/smtp-spa with the following contents, and chmod it 755 to make it executable, as shown in Example 9-2.

Example 9-2. Excessively simple Spamassassin SMTP-time filter

#!/bin/sh /usr/local/bin/spamassassin | /var/qmail/bin/qmail-queue

This works because qmail-queue first reads the body of the message from file descriptor 0, which is connected to the pipe, and then reads the envelope from file descriptor 1, which the shell doesn't change because there's no output redirection. Then add a line to the SMTP run script (line 2a in Example 9-3) that sets the environment variable.

Example 9-3. Running the SMTP daemon with an SMTP-time post-filter

 1. #!/bin/sh  2. limit datasize 40m 2a. export QMAILQUEUE=/var/qmail/bin/smtp-spa  3. exec \  4.    tcpserver -u000 -g000 -v -p -R    \  5.    -x/var/qmail/rules/smtprules.cdb 0 25 \  5a.        rblsmtpd -b \  5b.        -ahul.habeas.com \  5c.        -rsbl.spamhaus.org \  5d.        -rcbl.abuseat.org \  6.       /var/qmail/bin/qmail-smtpd 2>&1

Note line 2, where the data limit is increased to be large enough for Spamassassin to run. This example works, assuming you have Spamassassin installed, but there are two reasons that you probably want to set up a slightly more complex filtering script. One reason is that if Spamassassin fails, the mail disappears without a trace and qmail-queue delivers an empty message instead. The other reason is that Spamassassin has quite a few user-adjustable parameters, but when it's run here it has no access to the users' home directories where the parameter files live. The first problem is easily solved by using qmail-qfilter (described next) in the shell script to run the filter programs, but the only way to solve the second is to run at least some of the filtering code at delivery time if you let your users customize their filters.

It's possible to install extremely complex and slow spam and virus filters to run at SMTP time, but in my experience, you quickly reach the point of diminishing returns. Most users don't customize Spamassassin very much, so I find it reasonably effective to run Spamassasin at SMTP time, configured to add headers with a spam score, then use procmail at delivery time to look at the score and decide what to do with it.

To do SMTP-time filtering, you need at least two parts. One is the program that runs the filters, such as qmail-qfilter or qmail-scanner; the rest are the actual filters. This can go to a third level if you do virus filtering with amavis, which unpacks a message into its individual parts and then passes each part to a third-party filter.

While qmail-qfilter is a fairly simple program that runs a message through a few filters you specify before handing it to qmail-queue, qmail-scanner combines built-in message scanning with separate virus and spam filters to provide one-stop mail filtering (which is fine if their stop is the one you want).

9.5.2.1 Filtering with qmail-qfilter

You can download qmail-qfilter from http://untroubled.org/qmail-qfilter/, as either an RPM or a tar.gz file. Pay attention to the note in the README file that points out that it defines TMPDIR in qmail-qfilter.c, the place to put temporary copies of messages as /tmp, which you may want to change to /var/tmp or /usr/tmp if you have a small /tmp on a ramdisk. Compile and install it like any other application; it's a separate program, not a patch. To use it, you must apply the QMAILQUEUE patch to qmail, as described in Chapter 3.

When qmail-qfilter runs, it takes as arguments the names of the filters to which it will pass the mail message. If there's more than one program, they're separated by - -arguments. qmail-qfilter runs the filters one at a time, using temporary files to store the possibly modified message, and then if all of the filters succeeded (returned 0), passes the output of the last filter to qmail-queue. The QMAILQUEUE patch only passes the name of a command, so in practice qmail-qfilter is always run from a shell script. For example, Example 9-4 presents a script to run incoming mail through the popular Spamassassin spam filter using qmail-qfilter.

Example 9-4. Run incoming mail through a spam filter

#!/bin/sh # spam filter incoming mail exec /var/qmail/bin/qmail-qfilter \     /usr/local/bin/spamassassin

The version I use first runs mail through DCC, which counts similar messages to estimate bulkiness (see http://www.rhyolite.com/anti-spam/dcc/), and then through Spamassassin, which already contains rules to use the info that DCC adds to the mail header. DCC doesn't have built-in support for qmail, but I wrote qmaildcc (available as always at www.qmail.org), a small Perl script intended to be run from qmail-qfilter, which passes incoming mail through DCC and adds a header noting the DCC bulkiness score (see Example 9-5). DCC can optionally also perform greylisting, temporary rejection of mail from unfamiliar sources, on the theory that real mailers will retry later but spamware won't.

Example 9-5. Run incoming mail through DCC and a spam filter

#!/bin/sh # greylist, bulk count and spam filter incoming mail exec /var/qmail/bin/qmail-qfilter \      /usr/local/bin/qmaildcc -- \      /usr/local/bin/spamassassin

DCC can optionally whitelist based on incoming IP address. Because tcpserver provides the IP address in the environment variable TCPREMOTEIP, qmaildcc can pass it along to DCC. Note the -- separating the DCC arguments from the call to Spamassassin. If you use a separate virus filter like Clamav, add it here, too.

This script doesn't try very hard to do per-user customization. qmail-qfilter passes the list of recipient addresses to the filter programs as the environment variable QMAILRCPTS, and qmaildcc has a little code that checks to see whether an address is in a domain in locals, in which case it passes the mailbox to DCC as the username, or a domain in virtualdomains in which it passes the first component of the corresponding address as the username. However, this only works for the simplest user setups. For better per-user customization, see delivery-time filtering, described next.

To hook this script into qmail, the QMAILQUEUE variable needs to be set. Assuming the script is called dofilter, the easiest approach is to set it in the run script, as we already did. In line 2, increase the datasize to be big enough to run whatever filter software you're running. Spamassassin is pretty big, but 40 MB should be enough. As always, whenever you change the run script, svc -t to reread the script and restart tcpserver (see Example 9-6).

Example 9-6. Running the SMTP daemon with an SMTP-time post-filter

 1. #!/bin/sh  2. limit datasize 40m 2a. export QMAILQUEUE=/var/qmail/bin/dofilter  3. exec \  4.    tcpserver -u000 -g000 -v -p -R    \  5.    -x/var/qmail/rules/smtprules.cdb 0 25 \  5a.        rblsmtpd -b \  5b.        -a'hul.habeas.com' \  5c.        -r'sbl.spamhaus.org' \  5d.        -r'cbl.abuseat.org' \  6.       /var/qmail/bin/qmail-smtpd 2>&1

You can and probably should put some entries into the rules file to override QMAILQUEUE for some hosts, as shown in Example 9-7. For example, you probably don't want to spam filter injected outgoing mail, unless you have extremely unruly users, and there are probably some known friendly hosts that you can trust not to send unwanted mail, so there's no point in running slow spam filtering software on their mail.

Example 9-7. Sample smtprules line

# mail from a friend, no filtering needed 10.30.10.10:allow,QMAILQUEUE="/var/qmail/bin/qmail-queue" # allow relay from this host, but clean up the mail (see below) 127.:allow,RELAYCLIENT="",QMAILQUEUE="/var/qmail/bin/qinject" # allow relay from other hosts on this network 172.16.42.:allow,RELAYCLIENT="",QMAILQUEUE="/var/qmail/bin/qinject" 172.16.15-18.:allow,RELAYCLIENT="",QMAILQUEUE="/var/qmail/bin/qinject"

If your mail server handles injected mail, you can run the injected mail through new-inject to clean it up, like the @fixme trick described in Chapter 6, but without needing an extra trip through the delivery queue. Create a script called qinject (see Example 9-8).

Example 9-8. qinject: Using qmail-qfilter to clean up injected mail

#!/bin/sh # unset potentially wrong env vars unset MAILUSER; unset USER; unset LOGNAME; unset MAILNAME; unset NAME unset MAILNAME exec /var/qmail/bin/qmail-qfilter /var/qmail/bin/new-inject -n

This unsets the environment variables used to concoct a From: line, so that the mail won't seem to come from the qmail daemon user, then it runs the message through new-inject. qmail-qfilter sets QMAILUSER and QMAILHOST from the message's envelope sender, so new-inject can create a reasonable From: line if need be.

Create as many filtering scripts as you want and assign them to different hosts or sets of hosts via entries in the rules file. A plausible setup would be to do spam and virus filtering (if you don't use the EXE patch to qmail-smtpd) for incoming mail, just virus filtering for mail injected from local hosts that run Windows, and no filtering at all for mail injected locally or from other Unix systems.

In the previous examples, the filtering programs always accept the mail, but they don't have to. If a filter program returns a nonzero code, the code is passed back to qmail-smtpd, which returns an error message to the SMTP client. Useful return codes include 31 for a permanent "554 mail server permanently rejected message" error, 71 for a temporary "451 mail server temporarily rejected message" error, and 73 for a temporary "451 connection to mail server rejected" error. As a special case, return code 99 discards the message without returning an error.

9.5.2.2 Filtering with qmail-scanner

qmail-scanner is a large Perl script, also run via QMAILQUEUE, which runs mail through a gauntlet of tests and filters. Conceptually, qmail-scanner is simpler than using qmail-filter. You just download it from http://qmail-scanner.sourceforge.net/, run the configuration script as root which builds the Perl script to call the tools that are available, and plug it into QMAILQUEUE. In practice it's a little more complex.

Before you can use qmail-scanner, as well as the QMAILQUEUE patch, you must install maildrop (or at least the reformime program from maildrop), some Perl modules described in the README file, and all of the spam and virus filters you want it to call. You also need to create a separate user ID, usually called qscand, for it to run as. By default it "quarantines" incoming viruses in ~qspamd/Maildir, so you must remember to look there from time to time and clean it out.

qmail-scanner does work, but it's extremely slow (the suggested timeout in case something hangs is 20 minutes) and is overkill for most qmail systems.