7.4 SMTP Authorization and TLS Security

To use SMTP authorization with qmail, you must patch qmail-smtpd to handle the AUTH command for remote users to log into the server. Although AUTH lets remote users prove who they are, it doesn't provide any security against third parties snooping on the mail as it leaves whatever network the roaming users are on, nor does it provide security against port redirection, where a network connects you to their own SMTP server rather than the one you asked for. (AOL does port redirection, not for malicious purposes, but because it lets their users send out modest amounts of mail as roamers without needing to reconfigure their MUAs, while blocking blasts of spam and viruses.)

The transport-level security (TLS) extension provides an encrypted channel for SMTP sessions similar to that used by SSL secure web servers. TLS is based on certificates that include the host owner's name and address along with the hostname and an email address. Each certificate is in two parts, the private key, which needs to be kept secret, and everything else including the public key, which is not secret. For incoming SMTP sessions, SMTP clients start a secure session, verify the server's certificate and check that the hostname in the certificate matches the name of the host that the client thinks it's talking to. The client can optionally present a certificate to the server for which the server can make the same checks. The server can also use the address in the client certificate to authenticate the user instead of a separate AUTH step, as described later in Authenticating Client Hosts with TLS.

There's a combined patch for qmail-smtpd that adds both SMTP AUTH and TLS, and a doubly combined patch that adds SMTP AUTH and TLS, as well as the badrcptto anti-spam patch described in Chapter 9 and some extra logging (the version that I use). The two combined patches both add the same SMTP AUTH and TLS code, so they're the ones I describe here. These are the largest patches described in this book, which makes it more likely that they contain bugs. I've looked at the code and it appears OK to me, but if you're concerned about security, you should read through the patch you use yourself.

For SMTP AUTH, the setup involves setting up a login/password checking program to validate the authorization values that remote hosts present and adjusting the tcpserver invocation of qmail-smtpd. If you're using the qmail POP server, use the same password validator. Users generally only need to set an option in their MUAs to use AUTH on outgoing mail using the same userid/password pair they use for POP or IMAP.

TLS requires the openssl library (included with many but not all recent Unix-like systems) and a TLS certificate for the SMTP server. If you happen to have an SSL web server with the same name as the mail server, use the same certificate it uses. Otherwise, make a new certificate. All certificates are signed; you can sign yours yourself, but most MUAs expect server certificates to be signed by a certificate authority (CA) that vouches for the authenticity of the certificate. The MUA has a set of validation certificates from well-known CAs built-in (Outlook and Outlook Express share their list with Internet Explorer), and if the signature isn't from one of the authorities in the list, the MUA at least warns the user that the certificate isn't properly signed, and in many cases refuses to transfer any mail. There's generally some way for the MUA's user to tell the MUA to accept the self-signed certificate from the server. If you have very sophisticated users, you can set up your own miniature CA to sign your certificates and try to get your users to install your CA certificate into their MUA's well-known lists. Alternatively, you can pay one of the well-known CAs to sign your certificate, which costs between $50 and $300 depending on the CA. At this point, most TLS users are sophisticated enough to get their MUA to accept one self-signed certificate for the smarthost they use regularly, but if you plan to offer TLS to a less technical user community, your easiest course is to pay a well-known CA for a signature.

If this all sounds like more trouble than it's worth, build your patched qmail with the TLS code turned off, and worry about it later if your users ask for it.

Authenticating Client Hosts with TLS

It's possible, albeit quite cumbersome, to use TLS to identify client hosts that are allowed to relay. If the email address in a verified client certificate is listed in control/tlsclients, qmail-smtpd lets the client send mail to any address, as though RELAYCLIENT were set.

To do the verification, qmail-smtpd checks the signature on the client certificate against the signer certificate(s) in control/clientca.pem. Although it would be possible in principle to have all of the client certs signed by a commercial CA, at $50/cert it rapidly becomes expensive, so in practice you must set up your own tiny CA. Fortunately, the script CA.pl distributed with the openssl library lets you create a CA with a certificate you can put in clientca.pem. Then for each client host create a signing request for the host, sign it, and turn the signed certificate into the PKCS12 format that most PC MUAs need. See the manpage for CA.pl for details.

For most purposes, using SMTP AUTH over a TLS link is adequately secure, and is a lot easier to set up than making, distributing, and installing all those certificates.

7.4.1 Installing and Building SMTP AUTH and (Optionally) TLS

Download either of the combined SMTP AUTH/TLS patches (see www.qmail.org for links to the latest versions), and apply the patch.

The TLS code depends on the open source openssl library. If you want to use TLS and your system doesn't have a recent version of the openssl library (0.9.6 or later), download the source from http://www.openssl.org, and build and install it. The configuration and installation procedure, documented in its INSTALL file, is straightforward.

If you don't want to use TLS, edit conf-cc to remove the option to compile in the TLS code. The patch changes the first line of the file to something like this:

cc -O2 -DTLS -I/usr/local/ssl/include

To turn off TLS, change it back to:

cc -O2

(Because the compilation process uses only the first line of that file, add the simpler line in front of the patched one in case you want to try TLS later.)

Before you rebuild qmail, if you're using the standard checkpassword 0.90 or any other password checker that doesn't keep plain text passwords, you need to turn off one of the SMTP AUTH features. The patch supports three varieties of AUTH known as LOGIN, PLAIN, and CRAM-MD5. The CRAM-MD5 variety needs plain text passwords, so you must disable it if your password checker doesn't support them. (A modified checkpassword that supports CRAM-MD5 using a separate password file with plaintext passwords is available at http://members.elysium.pl/brush/cmd5checkpw.) To turn off CRAM-MD5, edit qmail-smtpd.c and around line 40 is a definition of AUTHCRAM:

#define AUTHCRAM

Comment it out:

/* #define AUTHCRAM */

Now make to rebuild the patched qmail.

If you're using TLS, you must install a certificate for TLS to use. To create a self-signed certificate, become super-user and make cert. It will ask for identifying information for the certificate, including the host's two-letter country code (US for the United States), state name, company name, common name, and email address. Most of the info is merely for display if someone checks the certificate, but the common name should be the SMTP server's hostname, and the email address a contact address for the server's manager. It will put the public and private keys into control/servercert.pem, and also link it to control/clientcert.pem for use in outgoing mail. This is all the setup you need if clients who use TLS are willing to tell their MUAs to accept self-signed certificates. If you want a certificate signed by a CA, use make cert-req instead. It puts a "certificate request" into req.pem, which you can submit to the CA. When the CA sends back the signed version, add that to the end of control/servercert.pem.

Whether or not you're using TLS, now become super-user and make setup check to install the patched qmail. (If you have the new configuration files set up, it's OK to install this over the running version of qmail because the AUTH and TLS features do nothing until someone tries to use them.)

To configure AUTH, the run file for the SMTP server needs three new arguments to qmail-smtpd: the server's hostname, the password checker, and a dummy program that the password checker can run, usually /bin/true. (See Example 7-6.)

Example 7-6. The SMTP listening script for incoming mail

 1. #!/bin/sh  2. limit datasize 2m  3. exec                                             \  4.    tcpserver -u000 -g000 -v -p -R with SMTP AUTH \  5.     -x/var/qmail/rules/smtprules.cdb 10.1.2.3 25 \  6.    /var/qmail/bin/qmail-smtpd                    \ 6a.    mail.example.com                              \ 6b.    checkpassword                                 \ 6c.    /bin/true 2>&1

The hostname in line 6a is used only for CRAM-MD5 authorization, but the argument has to be there regardless of whether CRAM is used. A common error is to leave out the hostname argument, making /bin/true the password checker, which means that any user/password pair will be accepted, making the server an open relay.^[3] After adding the new arguments, restart tcpserver with svc -t, and test out AUTH by sending mail from client MUAs with both valid and invalid logins, making sure that the invalid login is rejected. Because an invalid login doesn't end the SMTP session (it just doesn't authorize) you must address the test messages to an address that wouldn't be permitted without AUTH. If you're using TLS, test it from your favorite PC MUA. Just turn on the MUA option to use TLS on outgoing mail, send a message, observe that the MUA complains about the server's self-signed certificate, tell the MUA to accept it anyway, and check that the mail is delivered.

^[3] So don't do that.

On outgoing mail, qmail-remote with TLS turned on automatically starts a TLS session whenever a server announces that it has TLS available. If control/clientcert.pem exists, its contents are used as the client certificate in outgoing sessions. Normally, make cert links the clientcert file to the servercert file, but if you're sending TLS mail to a smarthost run by your ISP, the ISP may provide you a client certificate to use instead. If there are some SMTP servers to which mail should only be sent using TLS connections with signed certificates, create the directory control/tlshosts, and for each server, put the CA certificates of the allowable signers in control/tlshosts/hostname.pem. Usually all of the hosts share the same set of signers, so all of the .pem files are links to the same file. In practice, the only host that you're likely to verify this way is your ISP's smarthost.