13.4 Building POP Toasters
| A POP toaster is a system that provides POP mail service for a potentially large set of mailboxes. Rather than create each mailbox as a Unix user account, a POP toaster generally runs as a single user, puts all of the mailboxes in virtual domains, keeps its own database of usernames, and arranges for mail deliveries and POP/IMAP sessions to use that database for validation. The widely used vpopmail package (previously called vchkpw) is available from http://www.inter7.com/vpopmail.html. It provides all of the POP toaster functions, along with some nice additions, such as POP-before-SMTP relay validation for roaming users, database interfaces so the user information can be kept in a MySQL, Pgsql, or Oracle database, and a design that makes it straightforward to create clustered mail servers for added performance or reliability. At the time of this writing, the current version of vpopmail is 5.3.16. 13.4.1 Installing VpopmailVpopmail uses the conventional autoconf configuration scheme. Download it from http://www.inter7.com/vpopmail.html, and unpack it into a directory. Don't try to build it yet; you must create the vpopmail user ID first. It depends on ucspi-tcp (the package that contains tcpserver) so be sure you've installed that already, as described in Chapter 3. All of vpopmail's mailboxes and control files belong to the same Unix user. The usual user and group IDs are vpopmail and vcheckpw. If you can, create them with numeric user and group IDs of 89. (Some versions of FreeBSD may already have them defined as 89.) If your vpopmail setup expands to multiple machines, you want to have the same numeric IDs on all of them, and 89 is as good a number as any. Be sure that the disk partition on which you create the vpopmail home directory has sufficient space for all of the mail directories you plan to create. In some cases, you can move directories around later and use symlinks to splice the subtrees together, but you might as well allocate enough space in the first place and avoid the trouble. If you plan to have many thousands of mailboxes, you should put them on a separate partition. Since Maildirs put each message in a separate file, the average file size on a mail partition is smaller than on general purpose partitions, so you must build the partition with extra inodes. You can estimate that average messages are about 5K, so divide the size of the partition by 5K and allocate that many inodes. In the following examples, I use /var/vpopmail as the home directory for the mailboxes. It doesn't matter for vpopmail's configuration whether it's a partition mount point or not. Before you configure vpopmail, you have to make a few decisions:
To configure vpopmail, become the super-user (it needs to look at password files to figure out your mail setup), and in the vpopmail directory, run configure: # ./configure --enable-passwd \ --enable-roaming-users=y \ --enable-defaultquota=50000000 \ --enable-qmail-ext=y Leave out the options you're not using. The number after defaultquota is the default mailbox quota to use, in bytes. Quotas can also be written in the form 1000000S,400C to set limits on both the mailbox total size in bytes and maximum message count. After configure runs, leave super-user and type make (or gmake if your system's normal make program isn't GNU make) to build all the programs. Then become user again and make (or gmake) install. 13.4.2 Setting Up VpopmailTo get your mail going, create the virtual domain(s) the mailboxes will use, create user mailboxes in those domains, and start up the POP server to let people pick up their mail. 13.4.2.1 Creating virtual domainsUse vadddomain to create a domain. As the super-user: # cd ~vpopmail # bin/vadddomain myvirt.com topsecret The arguments are the name of the virtual domain, and the password to assign to the postmaster mailbox in that domain. A few options are available for unusual situations:
(See the manpages in ~vpopmail/doc.) When you create a vpopmail virtual domain, it creates a directory for the domain under ~vpopmail/domains, and (somewhat disconcertingly) automatically updates the files in /var/qmail/control and /var/qmail/users. It adds a line to /var/qmail/users/assign creating a qmail user with the same name as the domain, adds a line to /var/qmail/control/virtualdomains routing mail to the domain to the user it just created, and adds a line to rcpthosts or morercpthosts to accept mail for the domain. It doesn't do is to create DNS MX records to tell the world to send mail for the domain to your mail host. You have to do that yourself by editing zone files or your DNS server's equivalent. If you create a lot of domains at once, use the -O flag for all but the last vadddomain to speed up creation. In practice, it's unlikely that you'll create enough domains to worry about it. 13.4.2.2 Creating mailboxesYou create individual mailboxes with vadduser, running as either vpopmail or root. The arguments are the address of the mailbox to create and optionally the password. If you supply no password, it prompts you to type one. You can optionally supply a password on the command line, or let it generate a random one: $ vadduser able@myvirt.com dontguess $ vadduser -r baker@myvirt.com Random password: LMd%tusw Arguments include:
If you have a file with usernames and password pairs, it's easy enough to use a script to add them all: ( read lastuser lastpass while read user pass do vadduser -s $user $pass done # do the last one without -s to force database rebuild vadduser $lastuser $lastpass ) < userfile Once the database is rebuilt by a command without -s, the mailbox exists and is ready to use. You can change passwords or other mailbox parameters with vmoduser and delete mailboxes with vdeluser. Run each with no arguments to see what the options are. (There's an online manual, but it is way out of date.) 13.4.2.3 Starting the POP serviceStarting the vpopmail POP server is a minor variation on starting the regular qmail POP server, mostly involving replacing the standard password checker with the vpopmail version. If all of your POP-able accounts are managed by vpopmail, you can make another small change to run the whole POP server as the vpopmail user, which is somewhat more secure. Assuming you've already set up the POP listening script as described earlier in this chapter, the changes for vpopmail are minimal (see Example 13-5). Example 13-5. The listening script1. #!/bin/sh 2. limit datasize 2m 3. exec \ 4. /usr/local/bin/tcpserver \ 5. -HRv -l pop.example.com \ 6. -x /var/vpopmail/rules.cdb \ 7. 0 110 \ 8. /var/qmail/bin/qmail-popup pop.example.com \ 9. /var/vpopmail/bin/vchkpw \ 10. /var/qmail/bin/qmail-pop3d Maildir 2>&1 On line 6, move the rules.cdb file into the vpopmail home directory, and on line 9, use the vpopmail password checker rather than the standard one. If you've enabled roaming users, ~vpopmail/etc/tcp.smtp contains the fixed tcpserver relay rules, so put the info there to permit local users to relay; for example: # allow from localhost 127.:allow,RELAYCLIENT="" # allow from local network 10.1.2.:allow,RELAYCLIENT="" Then modify your SMTP listening script, which is in /var/qmail/supervise/qmail-smtp/run, so that it uses the vpopmail rules, as in Example 13-6. Example 13-6. Change the SMTP server rules5. -x/home/vpopmail/etc/tcp.smtp.cdb 0 25 \ Then start or restart your SMTP server, and you're all set. The /etc/passwd users can log in with their usernames and system passwords, while vpopmail users log in with their full email address, such as myname@myvirt.com and their vpopmail passwords. For the benefit of people using mail clients that don't like to put at-signs in their configuration data, vpopmail also accepts a percent or slash, for example, myname%myvirt.com or myname/myvirt.com. If you are only picking up vpopmail mailboxes, run the POP server as the vpopmail user, as shown in Example 13-7. Example 13-7. Change the POP server rules4. /usr/local/bin/tcpserver -u89 -g89 \ Substitute in the actual user and group IDs, if they're not both 89. This makes it somewhat more secure in the face of unexpected bugs. 13.4.2.4 Roaming usersVpopmail includes support for POP-before-SMTP that is very easy to set up. Assuming you've set up tcp.smtp as described in the previous section, vchkpw automatically adds each address that logs in for POP to ~vpopmail/etc/open-smtp and rebuilds tcp.smtp.cdb to include those addresses, and if you've modified your SMTP tcpserver commands as described previously, it lets POP-before-SMTP users relay mail. The only other thing you have to do is to run the clearopensmtp daemon that removes out of date entries from the open-smtp list. Run it from cron once an hour, with an entry like this in /etc/crontab: 0 * * * * root /var/vpopmail/bin/clearopensmtp If your POP server only picks up vpopmail mailboxes and runs as the vpopmail user, run clearopensmtp as vpopmail. Otherwise you have to run it as root, because vchkpw writes the open-smtp file as root. 13.4.3 Some Handy Vpopmail TricksAlthough vpopmail's normal setup delivers mail only to individual mailboxes, you can configure it to do just about anything that qmail can do. 13.4.3.1 Handling unknown usersWhen mail arrives for a mailbox that doesn't exist, it can bounce the message, discard it, or deliver it to a default mailbox either in the domain or elsewhere. When you create a domain with vadddomain, the -e flag sets the default, but it's easy to change later. In the domain's directory, usually ~vpopmail/domains/domainname, the .qmail-default line controlling deliveries to the domain looks like this: | /var/vpopmail/bin/vdelivermail '' defaultinstruction The second argument (the first, the two quotes, is just a placeholder) is the instruction to control the default behavior. If the instruction is the string bounce-no-mailbox, mail to undefined addresses bounces. If the instruction is the string delete, mail to undefined mailboxes is discarded. If the instruction is anything else, it's taken to be the email address to forward the mail to. If you want to deliver unknown mail to a default mailbox within the domain, you must write out the full address, e.g., catchall@myvirt.com. An unqualified address is treated as a local qmail address, which is rarely what you want. To change the default behavior, just edit .qmail-default as needed. 13.4.3.2 Forwarding a user's mail and other per-user special handlingFor each mailbox, vpopmail creates a directory with the same name as the mailbox and, in that directory, a Maildir. Usually the mailbox directory is in the domain's directory; in very large domains the directory is split up using intermediate directories named with small numbers. When vdelivermail, the vpopmail delivery program, delivers a message, it normally deposits it into the Maildir. But if the mailbox directory contains a .qmail file, vdelivermail processes that .qmail file using a subset of the qmail rules. Lines ending in /Maildir/ are taken to be Maildir names to which the message is delivered. Lines starting with a vertical bar are commands run by the shell. Anything else is treated as an email address, perhaps preceded by an ampersand. As a special case, if the mailbox's own address appears in the .qmail file, it's ignored to prevent mail loops. (Use ./Maildir/ to deliver the message.) Neither comment lines nor mbox deliveries are supported, but the features that are supported are quite adequate to handle forwarding to other addresses, vacation programs, and other common mailbox features. 13.4.3.3 Enforcing mail quotasVpopmail allows you to set a per-mailbox disk quota. If mail arrives for a mailbox that's over quota, it's bounced rather than delivered. You can customize the bounce message, and you can also arrange to deliver a warning message to a recipient when a mailbox is about to be over quota. When you build vpopmail, the --enable-defaultquota sets the quota for each mailbox. When you create a domain, the -q flag to vadddomain can override the quota for the postmaster mailbox, and when you create a mailbox, the -q flag to vadduser can override the quota for that user. You can change the quota for an individual user or a whole domain with vsetuserquota: $ vsetuserquota @myvirt.com 25M # 25 megs all user $ vsetuserquota fred@myvirt.com 20M # 20 megs for fred In each case, the quota can be a plain number that is the total size of messages in the mailbox; 10MS,999C, to set limits on the total size and count of messages; or the string NOQUOTA to turn off quota checking. In the message size limit, the letters K and M have the usual meaning.[2]
When a message arrives for a user who is over quota, it is bounced back to the sender. If a file named .over-quota.msg exists in the domain's directory or in ~vpopmail/domains, the contents of that file is used in the bounce message in place of user is over quota, permitting per-domain customized bounce text. In addition, if a file named .quotawarn.msg exists in either of those two places, its contents is delivered to the user. That file should be a complete mail message with headers, saying something like "your mailbox is over quota and you won't get any more mail until you delete some of it." The delivery program remembers when it's delivered the over-quota message and won't deliver it to a user more than once a day. 13.4.3.4 Mail bulletinsA useful service originated by the qpopper POP server is bulletins, messages sent to all users in a domain. Vpopmail provides vpopbull to deliver bulletins to all mailboxes in a domain or all mailboxes in all domains, by copying or linking the bulletin message into everyone's Maildir. To use it, create the bulletin in a file, formatted as an email message called something like bulletin1. (If you plan to copy rather than make links to the file, omit the To: line, because it'll be added to each copy automatically.) From: support@myvirt.com (Support) To: Myvirt Users :; Subject: Mail server interruption Due to a server upgrade, you won't be able to pick up your mail between 3:00 and 5:00 PM on Saturday. But if you come by the office, there'll be free beer. The Management Then distribute it. As vpopmail, run: $ vpopbull -s -f bulletin1 myvirt.com You can list one or more domains to distribute the bulletin to users in those domains, or no domain at all to distribute the bulletin to all vpopmail users. A few useful options include:
If you know that the total number of mailboxes is less than the per-file hard link limit on your system (usually 32767), -h will run the fastest. If you have more mailboxes than that, use -s. Only copy the file if you need to put each user's address on the To: line for example, to defeat overenthusiastic spam filters.
13.4.3.5 Storing user data in a databaseNormally, vpopmail stores the user info for each domain in a file called vpasswd in the domain's directory, compiled into vpasswd.cdb for fast access. For large mail setups, the user info can instead go into a SQL database. While this is slower than direct access to a CDB file, it's considerably more flexible, and makes it easier to distribute mail and POP service across multiple hosts. There are database interfaces for Pgsql, Oracle, Sybase, but by far, the most popular database used with vpopmail is MySQL, so that's the one covered here. Before building a MySQL version of vpopmail, you must define a MySQL database for vpopmail to use, along with a MySQL user that has full access to that database and a password for that user. Edit vmysql.h and put in the server name, username, and password into the definitions near the beginning of the file. There are two sets of definitions, one for the server from which to read data and the other for the server to which to send updates. Unless you turn on MySQL replication, the READ versions are ignored, but you might as well make them the same: /* Edit to match your set up */ #define MYSQL_UPDATE_SERVER "localhost" #define MYSQL_UPDATE_USER "vpopmail" #define MYSQL_UPDATE_PASSWD "verysecret" #define MYSQL_READ_SERVER "localhost" #define MYSQL_READ_USER "vpopmail" #define MYSQL_READ_PASSWD "verysecret" /* End of setup section*/ Vpopmail manages the database tables itself, but you must make one decision about how it should do so, the so-called "many-domains" option. Normally, when many-domains is on, it puts all the user information for all domains into a single table called vpopmail, with the key being the combination of the pw_name field, the mailbox name, and pw_domain, the domain name. If many-domains is off, vpopmail can make a table of the mailboxes in each domain, with the table having the same name as the domain,[3] and the table key is just the mailbox name. If you have a small number of domains each with a large number of mailboxes, turning off many-domains can save space because the domain names don't need to be stored in the database tables. Unless you have many thousands of mailboxes, accept the default that is to turn many-domains on.
Now become super-user and rerun the configuration script, adding --enable-mysql=y to the arguments. If the MySQL include files and shared library aren't in the default places, you must provide arguments to tell vpopmail where to look for them. Here's the configuration command for our system: # ./configure --enable-qmail-ext=y --enable-roaming-users=y \ --enable-defaultquota=50 --enable-passwd \ --enable-mysql=y \ --enable-incdir=/usr/local/mysql/include/mysql \ --enable-libdir=/usr/local/mysql/lib/mysql To turn off many-domains, add --enable-many-domains=n. Then rebuild and reinstall vpopmail as previously described. If you already have created virtual domains with mailboxes, the vconvert program can convert the CDB database into MySQL. Run it, giving it the list of domains to convert: $ vconvert -c -m myvirt.com ... The --c and --m flags say to convert from CDB to MySQL. The various vpopmail programs work the same as always. 13.4.3.6 Data replication and other MySQL tricksThe MySQL support in vpopmail is considerably more extensive than I have room to describe here. Using MySQL's database replication features, you can build multi-host systems with the SMTP server on one set of hosts and the POP servers on another set of hosts. See Chapter 17 for more details. The MySQL module also supports mailbox aliases, a pseudo-mailbox that is delivered to one or more real mailboxes. This is handy when a user changes his email address but wants to keep getting mail to the old address. You can use MySQL for logging, with an entry added for every POP login attempt, successful or not. And finally, with modest programming effort, you can add your own fields to the database tables (edit vmysql.h). Vpopmail won't use them, but your own programs can use them for other user maintenance purposes.
|
