9.5 Retrieving Mail Messages

As we've already seen, sites that connect intermittently to the Internet can complicate mail relaying and delivery. The central issue for such sites is the method for forcing mail to be sent and retrieved periodically in some automated way. Basically, the local queue needs to be flushed (e.g., via sendmail -q or postfix flush) when the connection is made, and mail for local users needs to be retrieved. (Of course, these two processes can be handled by different servers and so need not happen at the same time). Sending local mail is easily handled by adding the appropriate command to the connection script (or creating a script that activates the connection, flushes the queue, and then terminates the connection).

Retrieving mail can be performed manually via the SMTP ETRN command on remote servers that allow SMTP connections and support the enhanced SMTP protocol. Here is an example:

# telnet kevin.ahania.com 25 Trying 10.0.19.223... Connected to kevin. Escape character is `^]'. 220 kevin.ahania.com ESMTP Sendmail 8.11.0/8.11.0;  Mon, 16 Apr 2001 11:22:54 -0400 EHLO astarte 250 kevin.ahania.com Hello astarte ... ETRN mailhost.zoas.org

The final command requests mail for the specified host.

The fetchmail program, written by EricRaymond, provides automated mail retrieval capabilities. It is a powerful program that supports a variety of transport protocols and authentication mechanisms. It operates by retrieving messages from a remote mail server and sending them on to SMTP port 25 on the local system (or a specified remote system). As a result, to the transport agent, they look like normal incoming mail messages.

You will often need to ensure that localhost is included in the transport agent's list of allowed relay hosts for fetchmail to function properly.

The fetchmail command is the heart of the package. It is generally started at boot time via a command like this one:

fetchmail -d 900

This command starts the program in daemon mode and specifies that it will poll each remote mail server every 900 seconds (four times an hour). When the daemon is running, the fetchmail command (without arguments) wakes up the daemon and forces an immediate poll of all servers defined in the configuration file. Alternatively, you can specify hosts to poll by listing their names as arguments, as in this example:

# fetchmail mailer.notaol.com

This command polls the specified host immediately, determining connection information from its configuration file entry (discussed below). Alternatively, you can specify various connection parameters via command-line options (which override settings in the configuration file entry).

The fetchmail --quit command form kills the running daemon. You can also include -v or -v -v for verbose/ultra verbose output.

fetchmail's default configuration file is ~/.fetchmailrc (i.e., located in the home directory of the user who issues the fetchmail command, typically root). An alternate location may be specified with the FETCHMAILHOME environment variable or the -f command-line option. The configuration file must have the protection mode 600.

Table 9-10 lists the most important fetchmail configuration parameters, giving both the configuration file and command-line option forms.

Table 9-10. Important fetchmail parameters

Keyword	Meaning	Command-line option
set daemon seconds	Set the polling interval in seconds when fetchmail is run as a daemon.	-d
set logfile path	Enable logging to the specified file.	-L
set syslogset nosyslog	Use/don't use syslog for messages (fetchmail logs to the mail facility, using the info, alert, and err severity levels).	--syslog--nosyslog
defaults settings	Specify defaults for various settings.
poll host	Define a remote mail server.
proto protocol	Connection protocol to use (e.g., pop3, imap, etrn).	-p
user username[33]	User account on the remote server.	-u
is user(s)ato user(s)a	Corresponding local user account (the two keywords are synonymous).
password stringa	Password for the remote account.
auth scheme	Specify the authorization scheme in use: e.g., password, kerberos, kerberos_v5, ntlm, ssh, any (try various in turn).	-A
localdomains list	Domains to treat as local.
smtphost host(s)a	Send incoming mail to this host (or the first available host when the argument is a list). Hostname(s) may include an optional port number: host/port (the default is port 25).	-S host
limit bytesa	Limit message to this size (ignored by the ETRN protocol).	-l
keepanokeepa	Retain/don't retain downloaded messages on the server (using ETRN implies nokeep).	-k-K
flusha	Delete old messages from the server before fetching new ones (valid for POP3/IMAP protocols only).	-F
folder path(s)a	Specify remote mailbox path (valid for IMAP only).	-r path
preconnect commanda	Run this command before connecting.
postconnect commanda	Run this command after connecting.
plugin command	Use this command to make the server connection.	--plugin
skip host	Poll this host only when it is explicitly listed on the command line (e.g. fetchmail [options] host).
via host	Poll this DNS name; when used, the string following poll is treated just as a label.
interval n	Poll this site only on every nth poll (i.e., less frequently than normal).

^[33] These are user-related options, which must follow all server-related options (unmarked) in configuration file entries.

Here is a sample .fetchmailrc file:

set logfile /var/log/fetchmail.logset syslog defaults proto pop3 user "ispuser"    poll pop.essadm.org pass "password" poll mailer.notaol.org proto imap      user "rjchavez24" there has password "another-password"      and is chavez here poll getmail via pop.essadm.org proto etrn  poll poffice.ahanai.com proto imap auth ssh      plugin "ssh %h /usr/sbin/imapd"

The first section of this configuration file defines some global settings and provides defaults for some parameters for the entries that follow. In this case, logging messages go to the specified log file and also to the syslog facility. The default connection protocol is POP3, and the default user is ispuser.

The first poll entry defines a POP3 connection to pop.essadm.org, and the entry specifies the password for the ispuser account on the remote system.

The second poll entry defines an IMAP mail server (at mailer.notaol.org), to which the local host connects as user rjchavez24 (with the indicated password), corresponding to the local user chavez. In other words, this entry retrieves the mail for rjchavez24 from the specified server and delivers it to user chavez. Note that you can use multiple user keywords with an entry to retrieve mail for multiple users in a single operation.

The third poll entry also retrieves mail from pop.essadm.org (as did the first poll entry). Here the target is specified as getmail, which functions simply as an entry label (which can be referenced on the fetchmail command line), and the host to which to connect follows the via keyword. This entry specifies the ETRN protocol, so it will cause fetchmail to issue an SMTP ETRN command to the remote server on behalf of the local host.

The final entry illustrates the method for using ssh to connect to a remote mail server, in this case poffice.ahania.com.

In fetchmail configuration files, you will sometimes see entries like this one, which is designed to retrieve mail for multiple local users from a common ("multidrop") mailbox on the mail server:

poll pop.essadm.org proto pop3 localdomains zoas.org ahania.com   user "ispuser" pass "password" to trucks * here

This entry polls to pop.essadm.org using the POP3 protocol, mapping the remote user ispuser to the local user trucks and passing through all other users' mail to the local host (specified by the asterisk as the final entry in the to user list). However, you should be aware that this approach is prone to many sorts of problems: mail to mailing lists can end up being delivered to the account running fetchmail instead of local subscribed users, mail destined for blind-carbon-copied recipients may be lost, mail loops can arise the list goes on. The fetchmail documentation recommends that you use the ETRN protocol instead in such circumstances.

There are quite a few other fetchmail features that space constraints preclude discussing in detail. Consult Table 9-10 and the fetchmail documentation for more information.

The fetchmail package also includes a graphical configuration tool, fetchmailconf, which can make setting up configuration file entries easier for new fetchmail users. Figure 9-7 illustrates its novice-mode configuration dialogs; these specific settings would create entries similar to some that we looked at earlier.

Figure 9-7. The fetchmail configuration utility

The program also has an advanced configuration mode, which lets you set up entries that are as complex as you need them to be.