5.2 Command-Line Message Generation

A collection of messages to be sent may reside in files on local storage, or the messages may be generated on the fly by a UNIX process that then hands them off for delivery by sendmail via a socket or a pipe. Sometimes the latter approach is called "interactive" email delivery. While it technically has nothing to do with sendmail's "interactive" delivery mode, starting with version 8.12, this mode is generally preferred for performing mass mailings of this variety. In general, interactive mail delivery greatly resembles the processes involved in relaying email messages, so almost everything said so far about relaying applies to this situation as well. The main distinction is that not only is the relaying server under one's control, but the message originator, often called the "injector," is as well. This ability to control message creation and injection works to our advantage.

Running in interactive delivery mode with SuperSafe also set to interactive can vastly reduce disk operations. When these variables are set, the sendmail 8.12 process that accepts the connection attempts immediate redelivery, writing the message to disk only if the first redelivery attempt fails, assuming the message is short. Because the sending side knows the size of these messages, no good reason exists not to make sure that the buffering threshold of the MTA exceeds the size of all messages involved in this transaction. For mass mailings, the messages tend to be relatively compact for the obvious reasons. The details of this configuration were chronicled in Section 3.2.

Recall that the downside for interactive delivery mode is that it might cause the SMTP (or pipe) connection from the injector to remain open longer while delivery is attempted. When the injector resides on a local network and under our control, this issue shouldn't be a problem. On the client side, a single injector process must simply wait a little longer for the connection to close. In the meantime, a well-written injector should consume no CPU resources and almost no memory. The exception arises when the injector process must be serial that is, if we can't run more than one injector per machine at any one time. In this case, the duration of the connection between the injector and the relayer should be made as short as possible under almost any circumstances, and total system throughput certainly will increase if sendmail runs in background mode. As long as the requirement is in place, the relaying machines are unlikely to act as the bottleneck.

If the injector supports SMTP, why not have it make the connections with the email servers of the intended recipients directly? The answer is that while it is straightforward to write a program that hands email via SMTP off to a well-understood program such as sendmail for relaying, sending Internet email involves much more than implementing code to correctly handle a client SMTP session. Just scratching the surface, the injector would have to understand SMTP's DNS requirements and DSNs, implement a queueing mechanism for failed connections, and deal gracefully with other email servers' flawed SMTP implementations. Once one has accounted for these requirements, one has written half (the sending half) of an entire MTA. I can't recommend going to all of this trouble. It's a lot of work, and it's even more difficult than it first seems.

Whenever possible, the injector should sort messages by the destination domain so that a minimum number of SMTP connections and a minimal amount of qf file rewriting take place on the sending machine. Starting with sendmail version 8.12, the MTA may sort a list with multiple recipients by recipient domain before the message is queued, but it's always cheaper if this step occurs before the messages are handed off to sendmail. Also, it should be fairly obvious that if a single message will go to 9 recipients at a single domain and 11 recipients at another domain, it would be better to send the 9 bound for the single domain in one batch and the 11 bound for the other domain in another than to send the messages in two groups of 10 in any combination.

If a site runs sendmail version 8.12 using the default installation, the sendmail binary will not have been installed as set-user-ID root, but the main mail queue, /var/spool/mqueue, remains writable only by the root user. In these environments, if sendmail is invoked from the command line, the message will first be queued in a special queue, usually /var/spool/clientmqueue, and then the message will be sent via an SMTP session to the master sendmail daemon listening on port 25 of that machine. This daemon will also queue the message (typically in /var/spool/mqueue) before sending it to its final destination.

From a performance standpoint, this double-queueing of outbound email is inefficient. Where the performance of sending email from the command line is important, it will be desirable to eliminate the writes into clientmqueue. Version 8.12 of sendmail will attempt to contact the recipient's email server directly via SMTP, queueing the message only once, as long as two conditions are met:

1. sendmail must not find the submit.cf file, usually stored in /etc/mail.

2. sendmail must still be able to queue messages.

It is straightforward to make sure that the first condition is met. The second condition can be met in several ways. The sendmail program can be run by the root user, the binary can be installed as set-user-ID root, or the program can be run by naming a special queue to which the user invoking sendmail can write. For example:

 /usr/sbin/sendmail -oQ/home/npc/mqueue [e-mail] 

Of these three options, I'd recommend adopting the last one in most mass-mailing environments. The root user could also bypass the clientmqueue directory even if the submit.cf file exists by using the -Am flag on the sendmail command line:

 /usr/sbin/sendmail -Am [e-mail]