Providing Remote Access to Email
| < Day Day Up > |
Adding Spam and Virus ProtectionModern mail servers can no longer just send and deliver mail. They must now also examine the content of a message to determine whether it poses a threat or nuisance to users. Without effective spam and virus filtering, valid email communications are likely to be overlooked in mailboxes filled with garbage. Until recently, the only server-based solutions for spam and virus filtering were commercial packages that cost thousands of dollars. Today, two very popular open source applications are available that can quickly and easily add state-of-the-art protection to your Postfix MTA: ClamAV and SpamAssassin. ClamAV provides protection against all the latest viruses and features continuously updated virus definitions. SpamAssassin uses several techniques to identify and grade spam; it has become so popular that it forms the basis for several commercial systems, including the Barracuda Firewall. The "glue" that attaches ClamAV and SpamAssassin to Postfix is amavisd-new. amavisd-new provides content filtering based on a plug-in architecture that recognizes both ClamAV and SpamAssassin and incorporates them into the mail transfer process. Surprisingly, this is quite straightforward to set up. Installing SpamAssassinSpamAssassin is installed as a Perl module (Mail::SpamAssassin) using the CPAN (Comprehensive Perl Archive Network) method introduced in Chapter 18, "Developing Applications and Widgets Using Scripting Languages." If you're familiar with the process, you know that it is highly automated just follow this example. (If you haven't used CPAN before, you might be prompted to manually configure the system. Tell it no and let it automatically choose the best settings.) brezup:jray jray $ sudo cpan cpan shell -- CPAN exploration and modules installation (v1.7601) ReadLine support available (try 'install Bundle::CPAN') cpan> install Mail::SpamAssassin CPAN: Storable loaded ok LWP not available CPAN: Net::FTP loaded ok Running install for module Mail::SpamAssassin Running make for F/FE/FELICITY/Mail-SpamAssassin-3.0.2.tar.gz LWP not available Fetching with Net::FTP: ... You will be prompted to answer a few questions during the installation; for example: What email address or URL should be used in the suspected-spam report text for users who want more information on your filter installation? (In particular, ISPs should change this to a local Postmaster contact) default text: [the administrator of that system] jray@poisontooth.com Unless the requested information is directly related to your mail server installation, simply go with the default response by pressing the Return key. Eventually (after several minutes of tests, installation, and so forth), you should see this: /usr/bin/make install -- OK SpamAssassin is now installed. The main configuration file is located in /etc/mail/spamassassin. You now have two choices: either install one or more of the SpamAssassin helper applications or go on to configuring SpamAssassin as-is. SpamAssassin Helper ApplicationsSpamAssassin, by itself, is capable of using blacklists, header analysis, and learned content analysis to classify a message as spam or ham (not spam). It also can make use of a number of additional tools to further improve its accuracy. At present, three products can be used by SpamAssassin: Razor (http://razor.sourceforge.net/), Pyzor (http://pyzor.sourceforge.net/), and the Distributed Checksum Clearinghouse (http://www.rhyolite.com/anti-spam/dcc/). These add-ons work by cataloging fingerprints of known spam. Incoming messages are fingerprinted and compared to a central database of spam. If a match is found, the message is known to be spam. Because SpamAssassin understands these tools, the biggest challenge to using them is getting them on your system. Let's take a look at how to install Razor the most popular tool of the bunch. First, download both the agents and SDK packages from http://razor.sourceforge.net/. After they've been downloaded, unarchive them and enter the SDK directory: brezup:jray jray $ curl -O http://voxel.dl.sourceforge.net/sourceforge/razor Next, make and install the software with perl Makefile.PL followed by sudo make install: brezup:jray jray $ perl Makefile.PL Checking if your kit is complete... Looks good ... Writing Makefile for URI Writing Makefile for razor-agents-sdk brezup:jray jray $ sudo make install cp lib/Digest/HMAC_MD5.pm ../blib/lib/Digest/HMAC_MD5.pm cp lib/Digest/HMAC.pm ../blib/lib/Digest/HMAC.pm cp lib/Digest/HMAC_SHA1.pm ../blib/lib/Digest/HMAC_SHA1.pm Manifying ../blib/man3/Digest::HMAC_MD5.3pm ... Installing /usr/local/man/man3/URI::ldap.3pm Installing /usr/local/man/man3/URI::URL.3pm Installing /usr/local/man/man3/URI::WithBase.3pm ... Now, repeat the process for the second archive (razor-agents): brezup:jray jray $ curl -O http://voxel.dl.sourceforge.net/sourceforge/razor
After the process for the second archive has finished, type sudo razor-client and Razor should create the appropriate links for the razor-client utility: brezup:jray razor-agents-2.67 $ sudo razor-client Creating symlink razor-client <== /usr/bin/razor-check Creating symlink razor-client <== /usr/bin/razor-report Creating symlink razor-client <== /usr/bin/razor-revoke Creating symlink razor-client <== /usr/bin/razor-admin Now you're ready to finish the Razor setup by creating a sitewide account that it can use. Because the /etc/mail/spamassassin directory already exists, you can use it as the home directory for Razor. Enter the following commands to create its working directory: brezup:jray razor-agents-2.67 $ sudo razor-admin -home=/etc/mail/spamassassin/.razor Next, edit the /etc/mail/spamassassin/.razor/razor-agent.conf file and add the following line to the end of the file: razorhome = /etc/mail/spamassassin/.razor/ After you've saved your file, Razor has been configured, registered, and ready to go. Your next step (unless you want to install another helper) is to make a few configuration changes to SpamAssassin. Configuring SpamAssassinSpamAssassin is configured sitewide by editing the file /etc/mail/spamassassin/local.cf. Individual users can have ~/.spamassassin/local.cf files with their own custom settings as well. To get SpamAssassin prepared for protecting your server, you have to edit the sitewide configuration file and change a few settings. By default, /etc/mail/spamassassin/local.cf should look like this: # This is the right place to customize your installation of SpamAssassin. # # See 'perldoc Mail::SpamAssassin::Conf' for details of what can be # tweaked. # ########################################################################### # # rewrite_header Subject *****SPAM***** # report_safe 1 # trusted_networks 212.17.35. # lock_method flock Change the file to match the following sample. A full description of all of these options is available by typing perldoc Mail::SpamAssassin::Conf. The sample provided here should enable basic filtering through SpamAssassin's built-in mechanisms and Razor. # This is the right place to customize your installation of SpamAssassin. # # See 'perldoc Mail::SpamAssassin::Conf' for details of what can be # tweaked. # ########################################################################### # always_add_headers 1 use_bayes 1 bayes_path /etc/mail/spamassassin/bayes bayes_auto_learn 1 rbl_timeout 10 razor_timeout 20 razor_config /etc/mail/spamassassin/.razor/razor-agent.conf required_score 9.0 Training SpamAssassinAs part of its heuristic approach to determining what is and isn't spam, SpamAssassin must be taught to recognize valid and invalid email. The easiest way to do this is to collect folders of spam and ham, and process them using the sa-learn tool. For example, assume that you've dumped messages into two directories for processing. You can load the spam into SpamAssassin with the following command: sa-learn --spam -C /etc/mail/spamassassin --showdots --dir <spam path> Similarly, a folder of ham can be processed with sa-learn --ham -C /etc/mail/spamassassin --showdots --dir <ham path>
The key to good spam filtering is properly training SpamAssassin: Take the time to process as much mail of both the spam and ham varieties as possible. Eventually, the auto-learn feature (activated in our configuration file) will classify messages for you on its own, but to begin, you have to help out.
Testing SpamAssassinYou should now be able to test SpamAssassin on a message to verify that it is working. To do this, pipe an existing message to the command spamassassin -tD. This will generate quite a bit of debug output, but should eventually give you some results on how SpamAssassin graded your message: $ cat ~jray/Library/Mail/Mailboxes/INBOX.mbox | spamassassin -tD ... Content analysis details: (-0.4 points, 9.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.4 SUBJ_ALL_CAPS Subject is all capitals -2.8 ALL_TRUSTED Did not pass through any untrusted hosts 2.0 PYZOR_CHECK Listed in Pyzor (http://pyzor.sf.net/) This particular message was ranked 0.4 not spam (and it wasn't). To be considered spam, a message must rate at least a 9.0 as configured in the /etc/mail/spamassassin/local.cf file. Starting the SpamAssassin DaemonSpamAssassin generally works by running a daemon process, spamd, and feeding it messages through a client spamc. To finish your SpamAssassin installation, you should set spamd to launch at system startup by placing the following line in /etc/rc or a new StartupItem (refer to Chapter 16, "Managing System Services and Configuration," for more information or the StartupItem earlier in this chapter as an example): /usr/bin/spamd -d -m 50 -H /etc/mail/spamassassin Make sure that you've started spamd before continuing with the setup of amavisd. Installing ClamAVThe second piece of our filtering puzzle is the ClamAV antivirus utility. Unlike SpamAssassin, ClamAV is a breeze to install. Download the latest source code from http://www.clamav.net/, unarchive it, and enter the distribution directory: brezup:jray jray $ curl -O http://voxel.dl.sourceforge.net/sourceforge/clamav/clamav-0.80 Next, configure the compile with ./configure --sysconfdir=/etc --with-user=amavisd --with-group=amavisd this will prepare the software using the amavisd user that will be controlling ClamAV. brezup:jray clamav-0.80 $ ./configure --sysconfdir=/etc --with-user=amavisd Before compiling, you'll need to make a simple change to the source code. Within the distribution directory, edit the file libclamav/zziplib/zzip-conf.h. Look for the code block like this: #ifdef TARGET_OS_FREEBSD #include <sys/types.h> #endif Remove the #ifdef and #endif lines from the block, leaving only #include <sys/types.h> Save your changes and exit the editor.
Now compile with make and subsequently install with sudo make install: brezup:jray clamav-0.80 $ make make all-recursive Making all in libclamav if /bin/sh ../libtool --mode=compile gcc -DHAVE_CONFIG_H -I. -I. -I.. -I.. -I./zziplib -I./mspack -I/usr/local/include -g -O2 -MT scanners.lo -MD -MP -MF ".deps/scanners.Tpo" -c -o scanners.lo scanners.c; \ then mv -f ".deps/scanners.Tpo" ".deps/scanners.Plo"; else rm -f ".deps/scanners.Tpo"; exit 1; fi rm -f .libs/scanners.lo ... brezup:jray clamav-0.80 $ sudo make install Making install in libclamav test -z "/usr/local/lib" || /bin/sh ../mkinstalldirs "/usr/local/lib" /bin/sh ../libtool --mode=install /usr/bin/install -c 'libclamav.la' /usr/bin/install -c .libs/libclamav.lai /usr/local/lib/libclamav.la /usr/bin/install -c .libs/libclamav.a /usr/local/lib/libclamav.a ranlib /usr/local/lib/libclamav.a ... We're ready to start setting up ClamAV. Creating and Updating the Virus DefinitionsBefore you can test ClamAV, it has to be loaded with the latest virus definitions. The utility that manages this is called freshclam and should be installed in /usr/local/bin. Before freshclam can run, you must first make some changes to /etc/freshclam.conf. First, comment out the line at the top that reads Example by placing a pound (#) sign in front of it, like this: #Example Next, search for the line #DatabaseMirror db.XY.clamav.net Uncomment this line, and then replace XY in the hostname with the appropriate county code for your location. (A list is available at http://www.iana.org/cctld/cctld-whois.htm.) A server in the United States, for example, would use this: DatabaseMirror db.US.clamav.net Now, update the ClamAV definitions by running sudo /usr/local/bin/freshclam: brezup:jray jray $ sudo /usr/local/bin/freshclam ClamAV update process started at Thu Dec 30 18:51:10 2004 SECURITY WARNING: NO SUPPORT FOR DIGITAL SIGNATURES See the FAQ at http://www.clamav.net/faq.html for an explanation. Downloading main.cvd [*] main.cvd updated (version: 28, sigs: 26630, f-level: 3, builder: tomek) Downloading daily.cvd [*] daily.cvd updated (version: 646, sigs: 2329, f-level: 3, builder: tkojm) Database updated (28959 signatures) from db.US.clamav.net (66.111.55.10). Because you're obviously going to want to keep your virus definitions up to date, now is a good time to add an entry to /etc/crontab to run freshclam a few times each day. The ClamAV developers recommend adding a line something like this <n> * * * * root /usr/local/bin/freshclam --quiet where <n> is a number between 3 and 57 that isn't evenly divisible by ten. The ClamAV developers are apparently hoping to spread out traffic to their update servers as much as possible. Testing ClamAVYou're finally ready to test ClamAV. Enter the source distribution directory again and type /usr/local/bin/clamscan test/*. You should see something like this: brezup:jray clamav-0.80 $ /usr/local/bin/clamscan test/* test/README: OK test/clam-error.rar: RAR module failure test/clam-error.rar: OK test/clam.cab: ClamAV-Test-File FOUND test/clam.exe: ClamAV-Test-File FOUND test/clam.exe.bz2: ClamAV-Test-File FOUND test/clam.rar: ClamAV-Test-File FOUND test/clam.zip: ClamAV-Test-File FOUND test/mbox/debugm.c: OK ----------- SCAN SUMMARY ----------- Known viruses: 28959 Scanned directories: 1 Scanned files: 8 Infected files: 5 Data scanned: 0.00 MB I/O buffer size: 131072 bytes Time: 1.907 sec (0 m 1 s) You're now ready to finish the project and configure amavisd-new itself. Installing amavisd-newThe final piece of software we need virus- and spam-protecting Postfix is amavisd-new. As much as I'd like to say this is a piece of cake, it isn't. There are a number of little steps that, if not followed closely, will bungle the whole thing. To start, you'll need to upgrade Berkeley DB, which is out of date on the shipping version of Tiger. Let's begin. Updating Berkeley DBDownload the source distribution from http://www.sleepycat.com and unarchive it: brezup:jray jray $ curl -O ftp://sleepycat1.inetu.net/releases/db-4.3.27.NC.tar.gz brezup:jray jray $ tar zxf db-4.3.27.NC.tar.gz Next, cd into the build_unix directory within the distribution and execute ../dist/configure --prefix=/usr: brezup:jray jray $ cd db-4.3.27.NC/build_unix brezup:jray build $ ../dist/configure --prefix=/usr checking build system type... powerpc-apple-darwin8.0.0b2 checking host system type... powerpc-apple-darwin8.0.0b2 checking if building in the top-level or dist directories... no checking if --disable-cryptography option specified... no checking if --disable-hash option specified... no checking if --disable-queue option specified... no checking if --disable-replication option specified... no checking if --disable-statistics option specified... no checking if --disable-verify option specified... no ... After the configure process has finished, compile with make: brezup:jray build $ make /bin/sh ./libtool --mode=compile cc -c -I. -I../dist/.. mkdir .libs cc -c -I. -I../dist/.. -O2 ../dist/../mutex/mut_tas.c -DPIC -o .libs/mut_tas.o cc -c -I. -I../dist/.. -O2 ../dist/../mutex/mut_tas.c -o mut_tas.o >/dev/null 2>&1 ... Finally, install with sudo make install: brezup:jray build $ sudo make install Installing DB include files: /usr/include ... Installing DB library: /usr/lib ... cp -p .libs/libdb-4.3.dylib /usr/lib/libdb-4.3.dylib cp -p .libs/libdb-4.3.lai /usr/lib/libdb-4.3.la cp -p .libs/libdb-4.3.a /usr/lib/libdb-4.3.a ... You're now ready to install the dozen or so Perl modules that amavisd-new depends on. Installing the amavisd-new Perl Dependenciesamavisd is, like SpamAssassin, Perl-based. In fact, amavisd is a simple Perl script that is copied to your computer. Unfortunately, you'll have to install a number of additional Perl modules before amavisd will run. To make life easier, cpan can automate much of the process. Enter the following commands at a command prompt: brezup:jray jray $ sudo cpan cpan> install Archive::Tar Archive::Zip Compress::Zlib Convert::TNEF Convert::UUlib MIME: Perl will ask a number of questions during the installation, mostly about installing additional prerequisite modules. Accept the defaults and all should be well. This process will take quite some time, so be patient. Now you're ready for amavisd-new. Installing amavisd-newOkay, you're not quite ready. First, you need to prepare some directories that the amavisd user can write to. Execute the following commands to create a working area within /etc/mail/amavisd and to set up /var/virusmails, which will contain quarantined virus-infected messages: brezup:jray jray $ sudo mkdir -p /private/etc/mail/amavisd/db brezup:jray jray $ sudo mkdir -p /private/etc/mail/amavisd/tmp brezup:jray jray $ sudo mkdir -p /private/var/virusmails brezup:jray jray $ sudo chown -R amavisd:amavisd /private/etc/mail/amavisd brezup:jray jray $ sudo chown -R amavisd:amavisd /private/var/virusmails brezup:jray jray $ sudo chmod 750 /private/var/virusmails Now, download the amavisd-new distribution from http://www.ijs.si/software/amavisd/. Unarchive and enter the source directory: brezup:jray jray $ tar zxf amavisd-new-2.2.1.tar.gz brezup:jray jray $ cd amavisd-new-2.2.1 Because the software is distributed as a simple Perl script, your final installation step is to copy amavisd to /usr/local/bin and the amavisd.conf file to your /etc directory: brezup:jray amavisd-new-2.2.1 $ sudo cp amavisd /usr/local/bin brezup:jray amavisd-new-2.2.1 $ sudo cp amavisd.conf /etc Finally (and we really mean it this time!), edit the /etc/amavisd.conf file, and look for lines (noncontiguous) similar to these: $daemon_user = 'amavisd'; # (no default; customary: vscan or amavis) $daemon_group = 'amavisd'; # (no default; customary: vscan or amavis) $mydomain = '<your mail domain name>'; # a convenient default for other settings $MYHOME = '/etc/mail/amavisd'; # a convenient default for other settings $TEMPBASE = "$MYHOME/tmp"; # working directory, needs to be created manually $QUARANTINEDIR = '/var/virusmails'; Edit the lines to reflect the values shown here, except for $mydomain, which should be set to your mail server domain name. Congratulations, you've just completed the basic configuration of amavisd-new! Testing amavisd-newAside from the settings you just tweaked, amavisd does a good job of finding and configuring itself. Both SpamAssassin and ClamAV should be automatically detected and used by the amavisd script. To test this, execute amavisd debug as the user amavisd: brezup:jray jray $ sudo -u amavisd /usr/local/bin/amavisd debug Dec 30 20:09:27 pc-105.clonestem.loc /usr/local/bin/amavisd[17080]: starting. /usr/local/bin/amavisd at pc-105.clonestem.loc amavisd-new-2.2.1 (20041222), Unicode aware Dec 30 20:09:27 pc-105.clonestem.loc /usr/local/bin/amavisd[17080]: user=83, EUID: 83 (83); group=, EGID: 83 83 (83 83) Dec 30 20:09:27 pc-105.clonestem.loc /usr/local/bin/amavisd[17080]: Perl version 5.008004 ... Dec 30 20:09:28 pc-105.clonestem.loc /usr/local/bin/amavisd[17080]: Found secondary av scanner ClamAV-clamscan at /usr/local/bin/clamscan ... Dec 30 20:09:28 pc-105.clonestem.loc /usr/local/bin/amavisd[17080]: SpamControl: initializing Mail::SpamAssassin Dec 30 20:09:32 pc-105.clonestem.loc /usr/local/bin/amavisd[17080]: SpamControl: done ... As you can see, ClamAV is recognized as a secondary antivirus scanner, whereas SpamAssassin is identified for spam control. There's only one more step left: tying amavisd-new to Postfix.
Attaching amavisd-new to PostfixTo get the amavisd process communicating with Postfix, you should first configure amavisd to start at boot by placing it in /etc/rc, creating a new StartupItem (refer to Chapter 16), or placing it in the /System/Library/StartupItems/Postfix StartupItem. All you need to do to start amavisd is invoke it at the command line: brezup:jray jray $ sudo /usr/local/bin/amavisd After amavisd has been started, edit the Postfix file /etc/postfix/master.cf, and add the following lines (all of them!) to the end of the file: smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipent_maps= -o smtpd_restriction_classes= -o smtpd_client_restrictions= -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o mynetworks=127.0.0.0/8 -o strict_rfc821_envelopes=yes -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o receive_override_options=no_header_body_checks Save your changes, and then open /etc/postfix/main.cf and add this to the end of the file: content_filter=smtp-amavis:[127.0.0.1]:10024 Reload the Postfix configuration by typing sudo postfix reload. You can verify that Postfix and amavisd-new are working correctly by telneting to ports 10025 and 10024 on your server. You should see a banner message from each. Your server is now protected. To test it, send yourself a message. If everything is working as planned, you'll see a new header when you retrieve your email; something like this: X-Virus-Scanned: amavisd-new at poisontooth.com If a virus had been detected or the message graded as spam, it would be either quarantined in /var/virusmails, or bounced, respectively. Fine-tuning amavisd-new OperationTo fine-tune your server, you'll want to read the amavisd-new documentation and edit the /etc/amavisd.conf file. For example, these lines control spam grading: $sa_tag_level_deflt = 2.0; # add spam info headers if at, or above that level $sa_tag2_level_deflt = 6.31; # add 'spam detected' headers at that level $sa_kill_level_deflt = 6.31; # triggers spam evasive actions $sa_dsn_cutoff_level = 10; # spam level beyond which a DSN is not sent By default, if a message receives a grade of 2, spam headers are added to it. If it receives a score of 6.31 (their values, not mine), it will be bounced and not delivered. Finally, if the spam score is over 10, it will be discarded without a bounce. You might also want to change the usernames that are used to generate bounced messages, just in case a valid email is bounced and the recipient needs to reply. Alternatively, just create these user accounts or aliases on your server so that messages sent to them can be delivered. Again, edit these lines in /etc/amavisd.conf: $virus_admin = "virusalert\@$mydomain"; # notifications recip. $mailfrom_notify_admin = "virusalert\@$mydomain"; # notifications sender $mailfrom_notify_recip = "virusalert\@$mydomain"; # notifications sender $mailfrom_notify_spamadmin = "spam.police\@$mydomain"; # notifications sender There are hundreds of options to explore in SpamAssassin, ClamAV, and amavisd-new. We recommend that you familiarize yourself with these products before deploying a production server. The setup presented here should be a good start, but spam filtering is an art; it will take training, tweaks, and attention to deploy an efficient and effective system. |
/razor-agents-sdk-2.03.tar.gz| < Day Day Up > |
EAN: 2147483647
Pages: 251