| The SMTP servers described in this chapter accept mail for delivery from remote systems. To this point in the chapter, I haven't described what happens after the mail server accepts the message. In some cases, the mail server simply appends the message to a file in which a user 's incoming mail resides. Most Linux systems, though, come configured to use another tool, Procmail, for mail delivery. Procmail allows sophisticated processing of mail after it's been accepted by the MTA. You can set up a system-wide Procmail filter rule set, or individual users can design their own filters. There are also ready-made filter rule sets to accomplish specific tasks . Understanding how to use Procmail can allow you to accomplish a great deal with your mail delivery system that you could not otherwise do. Understanding the Role of Procmail Most MTAs are specialized to pass mail messages from one computer to another, or from one user to another on the same computer. Many mail environments, though, are complex enough that it's desirable to process or filter mail in some way once it's received. One reason for doing so has already been discussed: spam filtering. You might want a spam filter to discard all spam, or at least dump it into a special folder reserved for suspected spam. Sendmail, Exim, Postfix, and other MTAs all offer at least limited abilities to do filtering like this, but these MTA features are usually limited compared to those of Procmail. Exim comes closest to matching Procmail's features; Postfix can only filter on headers, and its action abilities are limited; and sendmail is still more limited in the headers it can search. Thus, Procmail fills a gap in mail filtering abilities. Procmail filters can do more than simply discard mail, as you might want to do with spam. You can use these filters to automatically sort mail into specific mail folders. For instance, if you subscribe to several mailing lists, you might prefer to have mail from your mailing lists sent to particular mail folders that you can read separately from your regular mail. You might check these folders once a day, rather than more frequently for your regular mail. Procmail can even pipe mail through other programs. One use for this feature is to create a mail-to -news gateway, in which mailing list messages are sent to a news server (described in Chapter 12, Running a News Server). You can then read your mailing list messages using a news reader rather than a mail reader. External programs could also process the mail in various ways. For instance, you could use Procmail to pass messages that contain attachments through a virus scanner to help protect computers on your network from viruses, worms, Trojan horses, and the like. You might set up a filter to play a certain sound file when messages that contain certain words arrive , to alert you to important e-mail's arrival. All of these features rely on Procmail's ability to scan messages. For instance, a Procmail-based spam filter might search for telltale signs that a message is spam. These might include Subject: headers that contain strings like $$$ or lines near the end of the message that reference S.1618 (a failed piece of U.S. legislation dealing with spam to which many spammers refer as if to legitimize their activities). To do this, Procmail uses regular expressions, similar to those used by Postfix in its header filters, or by the egrep program. I describe this in more detail shortly. Procmail can be used in two ways: as a system-wide filter or as a filter for individual accounts. For instance, you might want to implement a system-wide Procmail filter to block the most egregious forms of spam or to scan e-mail attachments for viruses. Individual users might want to use Procmail to sort incoming mail into folders for mailing lists, individuals, and so on. System-wide Procmail configuration is handled through the /etc/procmailrc configuration file, while individual Procmail filters appear in the .procmailrc file in the user's home directory. The format of both files is the same. WARNING  | The /etc/procmailrc configuration file is used when Procmail is run as root on most systems. Therefore, you should be very careful about what commands you have Procmail run in such a file, and if you redirect message delivery, you may have to take steps to ensure that the files created are readable by the desired users. The user .procmailrc files don't suffer from this problem; commands run from these files run with the user's privileges. |
A Procmail configuration file consists of three types of entries: -
Comments ” A Procmail comment begins with a pound sign ( # ) and continues until the end of the line, as with many configuration files. -
Environment variable assignments ” Procmail relies on certain environment variables , such as $HOME (the user's home directory) and $MAILDIR (the directory in which a user's mail folders reside) to work. You can adjust these and other environment variables much as you would in a shell script. For instance, MAILDIR = $HOME/Mail sets the $MAILDIR environment variable to the Mail directory in a user's home directory. -
Recipes ” A Procmail filter rule is referred to as a recipe. The bulk of the work you put into your filter set will probably be devoted to recipes. Each recipe provides rules for what to do with a message that matches one regular expression. Thus, a complete rule set is likely to consist of many recipes. Recipes are further subdivided into two classes: delivering and nondelivering. Delivering recipes cause mail to be delivered to a mailbox file, discarded, or piped through some other program. Nondelivering recipes cause the message to be processed again by Procmail, or nest one recipe within another. You can intersperse these three types of entries as required. Many Procmail configuration files begin by setting a handful of environment variables and then proceed with a series of recipes. In handling its configuration file, Procmail scans incoming mail and tries to match all its recipes against the mail. If none of the recipes match, Procmail delivers the mail to $DEFAULT , which is normally the default mailbox file, such as /var/spool/mail/ username . Designing a Recipe Procmail recipes look intimidating to the uninitiated, particularly to those who aren't already familiar with regular expressions as used by egrep or similar tools. The basic format of a recipe is as follows : :0 [ flags ] [:[ lockfile ]] [ conditions ] action This format can be considered to consist of three parts : the recipe identification line, the conditions, and the action. The Recipe Identification Line Every recipe begins with the string :0 . There's no special significance to the number , and you won't find recipes that begin with :1 or higher numbers . You can specify one or more of a variety of flags , which modify Procmail's behavior. You should consult the procmailrc man page for details, but the most common flags are: -
H ” This causes the pattern match to be done on the message headers. This is the default value. -
B ” This causes the pattern match to be done on the message body. -
D ” By default, the pattern match doesn't distinguish between upper- and lowercase characters . This option causes case to be significant. -
c ” This causes the recipe to work on a "carbon copy" of the original message; the "original" is left to match against other recipes. -
w ” This causes Procmail to wait for the action to complete. If the action doesn't finish successfully, the message remains in the queue to be matched against other recipes. -
W ” This works just like w , but suppresses program failure messages. After any flags, you can specify a single colon ( : ) to have Procmail create a lock file when processing the mail. A lock file is a special file that signals the fact that Procmail is working with another file. If Procmail sees that a lock file is present, it delays working with the message until the existing lock file is removed. This is an extremely desirable behavior, particularly on busy queues, to keep two messages delivered in quick succession from being written to a new file in an intermingled form. The default lock file is built based on the name of the destination mail file (as specified in the action line). If the action is a pipe to another program, you may want to specify a lock file by including the name after the colon. The Recipe Conditions The recipe conditions consists of zero or more lines that normally begin with asterisks ( * ). These are ordinary regular expressions, which in turn are strings that Procmail tries to match against lines in the input (the message header or body). Most characters match against their equivalents in the message body. There are, however, several special characters that match groups of strings or characters that are difficult to represent in a plain text file. These include the following: -
^ ” A carat signifies the start of a line. Many Procmail conditions lines begin with this character, after the leading asterisk. -
$ ” A dollar sign represents the end of a line. If you don't include this symbol, the conditions line matches any line that includes the specified string or other features, even if the line contains additional text. -
. ” A period matches any single character except for a newline. For instance, d.g matches dog , dig , dug , or any other three-letter string that begins with d and ends with g . -
a * ” This string matches any sequence of zero or more a s, where a can be any character. This is frequently used with a period as a , to match a string of characters of any length ”for instance, if you want to match a line that contains 802 followed by some unknown characters and then 1618 , you might use 802.*1618 to do the job. -
a + ” This works much like a * , but it matches one or more a characters, rather than zero or more. -
a ? ” This string matches zero or one a characters. -
seq1 seq2 ” You can specify a match to either of two possibilities by listing them both, separated by a vertical bar ( ). You can match against additional possibilities by using additional vertical bars. -
( seq )* ” This is essentially the same as a * , but it matches the string of characters seq rather than a single character. -
[ chars ] ” Characters enclosed in square braces are treated as a set, from which any one may match. For instance, [aeiou] matches any vowel character ( a , e , i , o , or u ). If one of the characters is a hyphen ( - ), any character between the surrounding characters matches. For instance, [m-q] matches m , n , o , p , or q . -
\ ” The backslash removes the special meaning from characters that possess such meanings. For instance, \. matches a single period. For more information on these expressions, consult the Procmail man page. It's possible to build up some very complex matching patterns using a combination of regular text and these special character sequences. (I present some examples shortly.) Remember that the recipe conditions is zero or more lines. Most recipes have at least one line. If a recipe has more than one line of conditions , they must all match if the recipe as a whole is to match. A zero-line conditions matches any mail message. It's possible to use special characters within a conditions line to change the interpretation of the recipe. These special characters include the following: -
! ” This is an inversion operator; a conditions line that begins with an exclamation mark indicates that the recipe matches if the pattern provided does not match. For instance, you might want to create a recipe that matches all mail except that which is addressed to postmaster . -
< ” The recipe's conditions are met if the length of the mail is less than the specified number of bytes. -
> ” The recipe's conditions are met if the length of the mail is greater than the specified number of bytes. The Recipe Action The Procmail action line is exactly one line that tells Procmail what to do with the message. A simple action line may consist of a filename (possibly including a variable), in which case Procmail stores the message in the specified file. This works well for sendmail, Exim, Postfix, and other mail servers that use the mbox mail folder format, in which an entire mail folder fits in one file. If you're using qmail or have reconfigured another mail server to use the maildir format for storing mail, you should be sure to include a trailing slash ( / ) on the action line to have Procmail store the message in a qmail-style maildir. Procmail also supports a third mail-storage format, specified by an action line that ends in a slash and period ( /. ). In addition to storing mail in a mail folder, Procmail supports several other actions, which you can specify by using special characters at the start of the action line: -
! ” If the action line begins with an exclamation mark, Procmail interprets the line as a list of e-mail addresses to which the message should be forwarded. You might use this feature if you want to automatically make certain message types available to a group of users. -
” A vertical bar is a pipe character in shells like Bash, and Procmail treats it in the same way. If an action line begins with this character, Procmail runs the specified program and pipes the message to the program. You might use this feature to scan a message in a more thorough way or to create an automated e-mail service that processes files. -
{ ” A left curly brace denotes the beginning of a nesting block. You can place additional recipes within this nesting block, and these recipes will only be used if the surrounding recipe's conditions are met. (The surrounding recipe is a nondelivering recipe, so if none of the enclosed recipes match, the message is not considered to be delivered.) You might use this feature if you have several recipes you only want to use if certain preconditions are met. For instance, you might use a nesting block that locates mail that shows certain inconclusive signs of being spam, then use recipes within the nesting block to search for other signs of the message being spam. A nesting block ends with a single right curly brace ( } ) on a line of its own. Procmail supports only one action per recipe. Thus, if you want to do multiple things with a message, you may need to create a script to do them all and then call the script through a pipe. If you do this, be sure that your script reads the entire message, or Procmail will attempt to match the message against other rules. In some cases, you might use the carbon copy ( c ) flag on all but one recipe, but this may result in multiple deliveries of the mail, depending upon what actions are performed in each recipe. Some Example Recipes The preceding discussion may seem fairly abstract, but it's necessary for you to understand a Procmail filter file. Listing 19.3 shows an example of such a file, albeit a fairly simple one. This example makes more sense as an individual's .procmailrc file than as a global /etc/procmailrc file, because it delivers some messages to folders within the user's home directory. Listing 19.3 shows several important Procmail recipe features. -
Nested recipes ” The spam-filter recipes are nested within a filter rule that causes the spam filters to be checked only if the mail was not addressed to postmaster . (Note the inversion operator on the outside nesting recipe.) You could produce a similar effect by including the *! ^To:.*postmaster condition in each of the individual spam filter recipes, and in this short example, this would produce a slightly shorter configuration file. In longer files, though, using a nested recipe can reduce the configuration file length. Nesting also reduces the chance for an error, because there's no need to copy the nesting condition to multiple recipes. Listing 19.3 A sample Procmail configuration file MAILDIR = $HOME/Mail # First, some spam checks, but don't ditch anything # addressed from or to postmaster :0 *! ^(FromTo):.*postmaster { :0 B * ^.*301.*S.*1618 /dev/null :0 * ^From:.*badspammer\.net /dev/null :0 * ^Subject:.*$$$ /dev/null } # Copy anything about rugs from david to amy :0 c * ^From:.*david@pangaea\.edu * ^Subject:.*rug ! amy@threeroomco.com # Put mail to mailing list in its own folder :0: * ^To:.*list@mailinglist\.example\.com $MAILDIR/mailinglist -
Spam conditions ” Listing 19.3 uses three spam checks. The first checks the message body (note the B flag) for a line that contains the strings 301 , S , and 1618 , in that order. This line is intended to catch references to the failed S.1618 bill, mentioned earlier. (These citations usually begin with a reference to section 301 of the bill.) The second spam filter blocks everything from the badspammer.net domain, and the final filter blocks everything that contains $$$ in its Subject: header. Note that both of these last two filters use the backslash ( \ ) quoting character. All three of the spam tests "save" the mail to /dev/null , which is a quick way for Procmail to get rid of a mail message. This is an unforgiving mail filter, though; once a message is sent to /dev/null , it cannot be retrieved. There's no need to use a lock file with these spam recipes, because the messages aren't being stored; they're being destroyed . -
Message copying ” Rather than write a message to a file, the second top-level recipe copies messages (note the c flag and the exclamation mark in the action line) to another user. The messages in this example must meet two criteria: They must be from david@pangaea.edu and they must contain the word rug in their Subject: headers. If either of these conditions is not met, the message isn't copied . -
Message sorting ” The final recipe sorts mail; it directs all mail addressed to list@mailinglist.example.com to its own folder in the user's mail directory. Many mailing lists use a To: message header of the mailing list itself; they use the envelope To header to deliver the mail to the individual recipients. You may need to examine the headers of mail on mailing lists to which you subscribe in order to find a reliable way of redirecting that mail to appropriate mail folders on your system. Listing 19.3 is a simple example that's unlikely to be useful to you directly, but you might be able to use it as a basis for something that would be suitable. Alternatively, you may want to try using or adapting a filter set you can obtain from somewhere else. Using Existing Filter Sets Creating a good Procmail filter set for any given purpose can be a tedious undertaking. There are several ready-made filter sets designed for specific purposes, and you might be able to adapt a filter set created by a friend or colleague more easily than you could write one from scratch. Some possible sources of Procmail filter sets include the following: -
The SpamBouncer ” This package is a set of Procmail filters designed to block spam. These filters are complex enough that they actually come as several files, and you'll probably have to edit some of these files to use the system. Consult the SpamBouncer documentation for details. You can download the package from its home page, http://www.spambouncer.org. -
SmartList ” This is a mailing list package built atop Procmail. You can read more about it on the SmartList FAQ, http://www.hartzler.net/smartlist/SmartList-FAQ.html. -
Timo's Tips and Recipes ” www . Timo Salmi has a Web page with Procmail tips and simple recipes at http://www.uwasa.fi/~ts/ info /proctips.html. This isn't a complete package in the sense that SpamBouncer or SmartList is a ready-made package, but it can be a good place to go to find out how to perform specific tasks with Procmail. -
Sample Procmail Recipes with Comments ” This Web site (http://handsonhowto.com/pmail102.html) presents a few Procmail recipes with comments describing their operation. A Web search on Procmail recipes is likely to turn up additional hits, or you can start from the Procmail Web site (http://www.procmail.org) and follow links until you locate a useful example filter set. Simple filter sets can be installed as .procmailrc files in users' home directories, or occasionally as system-wide /etc/procmailrc files. Some, such as SpamBouncer, rely on the presence of support files, so be sure to install them correctly. WARNING | | You shouldn't install a Procmail filter set blindly. Most require customization for your system, such as adjusting hostnames and even usernames. Many filter sets were designed to meet specific individuals' needs, and yours may differ . You might prefer to be more or less quick about sending suspected spam to /dev/null , for instance. |
TIP  | As a system administrator, you have the luxury of creating test user accounts. You can create such an account to test the action of a Procmail filter. You can create test messages to that account by altering the settings in a mail client program or even by using a Telnet program to reach the SMTP port (25) on the mail server program and typing envelope and message headers to simulate mail from various sources, as shown in Listing 19.2. |
Calling Procmail The preceding discussion assumes that Procmail magically runs and processes all incoming mail. This is normally true ”except for the "magically" part. Most Linux mail servers come configured to use Procmail for local mail delivery. This setting can be adjusted as follows: -
Sendmail ” There are three options that are commonly used to configure Procmail use in sendmail's m4 configuration file. The first is define(`PROCMAIL_MAILER_PATH',`/usr/bin/procmail') . This tells sendmail where to find the Procmail binary. The second and third lines are FEATURE(local_procmail) and MAILER(procmail) , which collectively tell sendmail to use Procmail for local deliveries. -
Exim ” About two thirds of the way through the default Debian exim.conf file is a configuration section labeled procmail_pipe . This section causes Exim to use Procmail for local mail delivery. If this doesn't seem to be happening, try searching for this section and verify that it's calling the correct binary file. -
Postfix ” The default Postfix configuration calls Procmail via the mailbox_command option in main.cf . If you omit this option, Postfix delivers mail directly, without involving Procmail. Chances are you won't need to adjust these options, because they're included by default in most Linux mail server setups. If by chance you're using a computer on which Procmail is not used by default, but you want to use it for mail deliveries to specific users' accounts, you can often enable it by creating a file called .forward in the home directories of the users who need to use Procmail. This file should contain the following line: "IFS=' '&&p=/usr/bin/procmail&&test -f $p&&exec $p -Yf-exit 75 \ # username " You should be sure to include the single and double quotes exactly as reproduced here. In this example, username is the username for the account that's to be processed.  |
