3.1 What Happens During Relaying

Consider a high-level view of what happens on a sendmail system that acts as an email gateway. First, an external system opens an SMTP connection with the gateway. The message is sent to the gateway, which writes it to local storage and then acknowledges (and accepts responsibility for) the message. At that point, the sending machine closes the SMTP connection and can delete its own copy of the message. The gateway next opens an SMTP connection to the message's true destination (or perhaps the next hop). It transmits the message to this machine, waiting to hear that the remote machine has accepted the message. At that point, the gateway may close its connection and remove the message from its queue.

Of course, the details of this operation are far more complex and have significant ramifications on the performance of the gateway machine. The following list offers additional detail, assuming we're using sendmail version 8.9 or earlier and delivery is being attempted in "background" mode, the default mode in which sendmail is started at system start-up time. After reviewing the items in this list, we will examine which of these steps changed as the sendmail version changed.

1. The originator machine opens an SMTP connection with the gateway.

2. The gateway accepts the connection, and the master sendmail daemon spawns a new sendmail process to handle this connection.

3. The new sendmail process inspects the SMTP envelope (the envelope is distinct from the email message's header; see RFC 2821 [KLE01] or the sendmail book [CA97] for more details) to determine whether the message should be accepted by policy.

4. If the message is acceptable to the gateway, a qf file, which will eventually contain the message's header along with delivery information, is created in the queue, typically the /var/spool/mqueue directory. Next, the remote machine receives permission to send the message itself, and then the gateway creates the df file, which will hold the message's body, also in the queue.

5. The originator sends the message's header, which the gateway stores in memory temporarily, followed by the message body. As the body is received by the gateway, it is buffered and written out to the df file. This procedure ensures that the sendmail machine can handle a message larger than its virtual memory system, if desired.

6. Once the originator indicates that the message has been completely sent, the gateway adds its own "Received:" line to the header, modifies the message headers if necessary, and writes out the contents of the qf file. It then issues an fsync() command on the still-open qf and df file descriptors, closes the files, and acknowledges receipt of the message to the originator.

7. The sendmail process itself spawns a child whose process memory data segment inherits the memory image of the qf file from the parent process.

8. The originator of the now-queued message closes the SMTP connection if no more messages will be sent, and the sendmail process that queued the message exits.

9. By this time, the new child process has begun parsing the data that were written to the qf file to determine what to do with the message.

10. Once the destination machine has been determined, an SMTP connection is opened to the appropriate machine and an xf file is created in the queue, in which the MTA will log error messages associated with each delivery attempt.

11. The message envelope information is sent to the destination machine, which decides whether it will accept or reject the message.

12. Once the message envelope has been accepted by the destination machine, the message header is generated from the contents of the qf file held in memory and sent over the SMTP connection, followed by a blank line and then the body of the message, which is read out of the df file.

13. If the message cannot be sent to at least one intended recipient, the MTA creates a tf file. In this file it writes an updated version of the qf file, fsync()s the data, rename()s the file to atomically replace the old qf file.

14. Once the message is successfully delivered to all recipients, then sendmail can unlink the df files and then the qf files. If a tf file has been created, an exiting sendmail process will rename it to a qf file. The only time a tf or xf file might be left around in a mail queue is if the sendmail process that was working on the file crashes or if the entire machine stops functioning before cleanup can occur. Of course, in all circumstances, an intact qf file will remain present in the queue, so no email will be lost, although under certain rare circumstances a recipient might be sent the same message more than once.

Starting with sendmail 8.10, some minor differences occur to the sequence listed above. With this version of sendmail, the queue identifier for a given transaction will be guaranteed to be unique for more than 60 years. Needless to say, no item should stay in a mail queue for that long. Therefore, there is no need to create a qf file to reserve the queue ID, so creating and writing to this file are deferred until just before the MTA acknowledges the receipt of the message. When sendmail runs in background mode (the default case, signified by adding the -bd flag when starting sendmail), this activity does not typically result in the net saving of any I/O operations.

Besides the filesystem and network operations described previously, sendmail performs several DNS operations during this exchange. Typically, a reverse DNS lookup is performed on the IP address of the connecting host to check whether that host has permission to connect. Another DNS lookup is performed on the response to see whether a DNS A or IPv6 AAAA record exists that matches the PTR record returned in the previous request. A DNS lookup is then performed on the sender address. Next, a DNS lookup is performed on each recipient address to determine the machine for which the message is destined (it might be destined for the gateway itself) and to canonify the address (replace CNAME records with the host names to which they point). Once the sendmail process attempts to deliver the email message, the destination host is checked in DNS again. Each of these DNS checks of domain names are first checks for MX records, and then checks for CNAME and A records and possibly AAAA records if IPv6 support is enabled. Thus, if MX records for the destination server don't exist, two DNS lookups will be performed. For the typical relayed message bound for a single recipient, at least four DNS checks will occur, each of which may consist of two distinct queries.

Matters become simpler if the sendmail server's name server is located nearby. If a DNS server receives a request to answer a query about, say, an MX record, the name server will typically tack on a request to resolve any resource records to which the MX record points, on the theory that the additional information will be requested next. While this procedure doesn't reduce the number of requests between an application and its name server, it does reduce the number of requests sent between the application's name server and the authoritative repository for the DNS information. Communication between a site's DNS server and the rest of the Internet typically occurs much more slowly than communication between the application and its name server, especially if an email server is its own name server.