How Email Is Sent and Received

Email is transmitted as plain text across networks around the world using the SMTP protocol (Simple Mail Transfer Protocol). As the name implies, the protocol itself is fairly basic, and it has been extended to add further authentication and error reporting/messaging in order to satisfy the growing demands of modern email. Mail transfer agents, or MTAs, work in the background transferring email from server to server allowing emails to be sent all over the world. You might have come across such MTA software such as Sendmail, Postfix, Fetchmail, Exim, or Qmail.

SMTP allows each computer that the email passes through to forward it in the right direction to the final destination. When you consider that there are millions of email servers across the world, you have to marvel at how simple it all seems.

Here is a simplified example of how email is successfully processed and sent to its destination:

1. [e-mail] composes and sends an email message to [e-mail].

2. The MTA at hudson.org receives andrew's email message and queues it for delivery behind any other messages that are also waiting to go out.

3. The MTA at hudson.org contacts the MTA at hudzilla.org on port 25. After hudzilla.org acknowledges the connection, the MTA at hudson.org sends the mail message. After hudzilla.org accepts and acknowledges receipt of the message, the connection is closed.

4. The MTA at hudzilla.org places the mail message into paul's incoming mailbox; paul is notified that he has new mail the next time he logs on.

Of course, several things can go wrong during this process. Here are a few examples:

What if paul does not exist at hudzilla.org? In this case, the MTA at hudzilla.org rejects the email and notifies the MTA at hudson.org of what the problem is. The MTA at hudson.org then generates an email message and sends it to [e-mail], informing him that no paul exists at hudzilla.org (or perhaps just silently discards the message and gives the sender no indication of the problem, depending on how the email server is configured).

What happens if hudzilla.org doesn't respond to hudson.org's connection attempts? (Perhaps the server is down for maintenance.) The MTA at hudson.org notifies the sender that the initial delivery attempt has failed. Further attempts will be made at intervals decided by the server administrator until the deadline is reached, and the sender will be notified that the mail is undeliverable.

The Mail Transport Agent

There are several MTAs that are available for Fedora, each with pros and cons to being used. Normally they are hidden under the skin of Fedora, silently moving mail between servers all over the world with need for little or no maintenance. Some MTAs are extremely powerful, and are able to cope with hundreds of thousands of messages each day, whereas others are more geared toward smaller installations. Other MTAs are perhaps not as powerful, but are packed full with features. In the next section, we will take a look at some of the more popular MTAs available for Fedora.

Sendmail

The overwhelming majority of emails transmitted over the Internet today is handled by Sendmail, which just so happens to be the default MTA supplied with Fedora. It is extremely popular across the Linux/UNIX/BSD world and is very well supported. There is a commercial version available, which has a GUI interface for ease of configuration.

As well as being popular, Sendmail is particularly powerful compared to some of the other MTAs. However, it is not without its downsides and other MTAs can handle more email per second in a larger environment. The other issue with Sendmail is that it can be extremely complicated to set it up exactly as you want it. Thankfully there are a few books available specifically for Sendmail, but the most popular one has more than a thousand pages, reflecting the complex nature of Sendmail configuration.

Thankfully the default configuration for Sendmail works fine for most basic installations out of the box, making further configurations unnecessary. Even if you want to use it as a basic email server, you have to do only some minor tweaks. The level of complexity associated with Sendmail often leads to system administrators replacing it with one of the other alternatives that is easier to configure. We will take a look at some basic Sendmail configuration later on in this chapter in the section titled "Basic Sendmail Configuration and Operation."

Postfix

Postfix has its origins as the IBM Secure Mailer, but was released to the community by IBM. Compared to Sendmail it is much easier to administer, and has a number of speed advantages. Postfix offers a pain-free replacement for Sendmail, and you are able to literally replace Sendmail with Postfix without the system breaking a sweat. In fact, applications that rely on Sendmail will automatically use Postfix instead and carry on working correctly. This is due to Postfix using a Sendmail-wrapper, which deceives other programs into thinking that Postfix is Sendmail. This wrapper, or more correctly, interface, makes switching to Postfix extremely easy.

Caution

Fedora provides Postfix version 2.2, which uses a slightly different configuration than the earlier version. If you are upgrading Postfix from an earlier Fedora or Red Hat version, check your configuration files.

Fedora also now compiles Postfix and Sendmail against version 2 of the Cyrus SASL library (an authentication library). The Release Notes contain detailed information on file location and option changes that affect you if you use these libraries.

For enhanced security, many Postfix processes used to use the chroot facility (which restricts access to only specific parts of the file system) for improved security, and there are no setuid components in Postfix. With the current release of Fedora, a chroot configuration is no longer used and is, in fact, discouraged by the Postfix author. You can manually reconfigure Postfix to a chroot configuration, but that is no longer supported by Fedora.

If you are starting from scratch, Postfix is considered a better choice than Sendmail.

Qmail and Exim

Qmail is a direct competitor to Postfix but is not provided with Fedora. Postfix is designed to be easier to use than Sendmail, as well as faster and more secure. However, Qmail isn't a drop-in replacement for Sendmail, so migrating an existing Sendmail installation to Qmail is not quite as simple as migrating from Sendmail to Postfix. Qmail is relatively easy to administer, and it integrates with a number of software add-ons, including web mail systems and POP3 servers. Qmail is available from http://www.qmail.org/.

Exim is yet another MTA, and it is available at http://www.exim.org/. Exim is considered faster and more secure than Sendmail or Postfix, but is much different to configure than either of those. Exim and Qmail use the maildir format rather than mbox, so both are considered "NFS safe" (see the following sidebar).

MDIR Versus Mailbox

Qmail also introduced maildir, which is an alternative to the standard UNIX method of storing incoming mail. maildir is a more versatile system of handling incoming email, but it requires your email clients to be reconfigured, and it is not compatible with the traditional UNIX way of storing incoming mail. You have to use mail programs that recognize the maildir format. (Modern programs do.)

The traditional mbox format keeps all mail assigned to a folder concatenated as a single file and maintains an index of individual emails. With maildir, each mail folder has three subfolders: /cur, /new, and /tmp. Each email is kept in a separate, unique file. If you are running a mail server for a large number of people, you should select a file system that can efficiently handle a large number of small files.

mbox does offer one major disadvantage. While you are accessing the monolithic mbox file that contains all your email, suppose that some type of corruption occurs, either to the file itself or to the index. Recovery from this problem can be difficult. The mbox files are especially prone to problems if the files are being accessed over a network and can result in file corruption; you should avoid accessing mbox mail mounted over NFS, the network file system, because file corruption can occur.

Depending on how you access your mail, maildir does permit the simultaneous access of maildir files by multiple applications; mbox does not.

The choice of a mail user agent, or email client, also affects your choice of mail directory format. For example, the pine program does not cache any directory information and must reread the mail directory any time it accesses it. If you are using pine, maildir would be a poor choice.

More advanced email clients perform caching, so maildir might be a good choice, although the email client cache can get out of synchronization. It seems that there is no perfect choice.

Fedora provides you with mail alternatives that have both strong and weak points. Be aware of the differences among the alternatives and frequently reevaluate your selection to make certain that it is the best one for your circumstances.

Choosing an MTA

Other MTAs are available for use with Fedora, but those discussed in the preceding sections are the most popular. Which one should you choose? That depends on what you need to do. Sendmail's main strengths are that it is considered the standard and it can do things that many other MTAs cannot. However, if ease of use or speed is a concern to you, you might want to consider replacing Sendmail with Postfix, Exim, or Qmail. Because Sendmail is the default MTA included with Fedora, it is covered in more detail in the following section.

The Mail Delivery Agent

SMTP is a server-to-server protocol that was designed to deliver mail to systems that are always connected to the Internet. Dial-up systems connect only at the user's command; they connect for specific operations, and are frequently disconnected. To accommodate this difference, many mail systems also include a mail delivery agent, or MDA. The MDA transfers mail to systems without permanent Internet connections. An MDA is similar to an MTA (see the following note), but does not handle deliveries between systems and does not provide an interface to the user.

Note

Procmail and Spamassassin are examples of MTAs; both provide filtering services to the MTA while they store messages locally and then make them available to the MUA or email client for reading by the user.

The MDA uses the POP3 or IMAP protocols for this process. In a manner similar to a post office box at the post office, POP3 and IMAP implement a "store and forward" process that alleviates the need to maintain a local mail server if all you want to do is read your mail. For example, dial-up Internet users can intermittently connect to their ISP's mail server to retrieve mail using Fetchmailthe MDA provided by Fedora (see the section "Using Fetchmail to Retrieve Mail," later in this chapter).

The Mail User Agent

The mail user agent, or MUA, is another necessary part of the email system. The MUA is a mail client, or mail reader, that allows the user to read and compose email and provides the user interface. (It is the email application itself that most users are familiar with as "email.") Some popular UNIX command-line MUAs are elm, pine, and mutt. Fedora also provides modern GUI MUAs: Evolution, Thunderbird, Mozilla Mail, Balsa, Sylpheed, and KMail. For comparison, common non-UNIX MUAs are Microsoft Outlook, Outlook Express, Pegasus, Eudora, and Netscape Messenger.

The Microsoft Windows and Macintosh MUAs often include some MTA functionality; UNIX does not. For example, Microsoft Outlook can connect to your Internet provider's mail server to send messages. On the other hand, UNIX MUAs generally rely on an external MTA such as Sendmail. This might seem like a needlessly complicated way to do things, and it is if used to connect a single user to her ISP. For any other situation, however, using an external MTA allows you much greater flexibility because you can use any number of external programs to handle and process your email functions and customize the service. Having the process handled by different applications gives you great control over how you provide email service to users on your network, as well as to individual and SOHO users.

For example, you could

• Use Evolution to read and compose mail

• Use Sendmail to send your mail

• Use xbiff to notify you when you have new mail

• Use Fetchmail to retrieve your mail from a remote mail server

• Use Procmail to automatically sort your incoming mail based on sender, subject, or many other variables

• Use Spamassassin to eliminate the unwanted messages before you read them