Chapter 1. Introducing SpamAssassin

The SpamAssassin system is software for analyzing email messages, determining how likely they are to be spam, and reporting its conclusions. It is a rule-based system that compares different parts of email messages with a large set of rules. Each rule adds or removes points from a message's spam score. A message with a high enough score is reported to be spam.

SpamAssassin was a trademark of Deersoft, and Deersoft has been acquired by Network Associates. In this book, I won't write SpamAssassin each time I mention it because that would be distracting, but you should assume that the trademark symbol is there.

Many spam-checking systems are available. SpamAssassin has become popular for several reasons:

• It uses a large number of different kinds of rules and weights them according to their diagnosticity. Rules that have been demonstrated to be more effective at discriminating spam from non-spam email are given higher weightings.

• It is easy to tune the scores associated with each rule or to add new rules based on regular expressions.

• SpamAssassin can adapt to each system's email environment, learning to recognize which senders are to be trusted and to identify new kinds of spam.

• It can report spam to several different spam clearinghouses and can be configured to create spam traps ‚ email addresses that are used only to forward spam to a clearinghouse.

• It is free software, distributed under either the GNU Public License or the Artistic License. Either license allows users to freely modify the software and redistribute their modifications under the same terms.

Example 1-1 shows a message that has been tagged as spam by SpamAssassin. Elements added by SpamAssassin appear in bold.

Example 1-1. A message tagged by SpamAssassin

 From [e-mail] Fri Nov  7 18:26:05 2003 Received: from localhost [127.0.0.1] by localhost         with SpamAssassin (2.60 1.212-2003-09-23-exp);         Sun, 09 Nov 2003 12:24:22 -0600 From: "brianj" <[e-mail]> To: <[e-mail]> Subject: Live your dream life!!                MPNWSTU Date: Fri, 07 Nov 2003 15:32:41 -0800 Message-Id: <000016646728 
 From [e-mail] Fri Nov 7 18:26:05 2003 Received: from localhost [127.0.0.1] by localhost with SpamAssassin (2.60 1.212-2003-09-23-exp); Sun, 09 Nov 2003 12:24:22 -0600 From: "brianj" <[e-mail]> To: <[e-mail]> Subject: Live your dream life!! MPNWSTU Date: Fri, 07 Nov 2003 15:32:41 -0800 Message-Id: <000016646728$00007347$[e-mail]>  X-Spam-Status: Yes, hits=12.9 required=5.0 tests=CLICK_BELOW, FORGED_MUA_EUDORA,FROM_ENDS_IN_NUMS,MISSING_OUTLOOK_NAME, MSGID_OUTLOOK_INVALID,MSGID_SPAM_ZEROES,NORMAL_HTTP_TO_IP, SUBJ_HAS_SPACES,SUBJ_HAS_UNIQ_ID autolearn=no version=2.60 X-Spam-Flag: YES X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) X-Spam-Level: ************  MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------=_3FAE8656.371BED4D" This is a multi-part message in MIME format. ------------=_3FAE8656.371BED4D  Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 8bit Spam detection software, running on the system has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or block similar future email. If you have any questions, see the administrator of that system for details. Content preview: Do you owe large sums of money? Are you stuck with high interest ra{tes? We can help! You can do what tens of thousands of americans have done, consolidate your high interest bills into one easy, low interest, monthly payment. [...] Content analysis details: (12.9 points, 5.0 required) pts rule name description ---- ---------------------- ---------------------------------------------- 1.0 SUBJ_HAS_SPACES Subject contains lots of white space 4.3 MSGID_SPAM_ZEROES Spam tool Message-Id: (12-zeroes variant) 0.9 FROM_ENDS_IN_NUMS From: ends in numbers 0.2 NORMAL_HTTP_TO_IP URI: Uses a dotted -decimal IP address in URL 0.2 SUBJ_HAS_UNIQ_ID Subject contains a unique ID 4.3 MSGID_OUTLOOK_INVALID Message-Id is fake (in Outlook Express format) 0.1 MISSING_OUTLOOK_NAME Message looks like Outlook, but isn't 1.9 FORGED_MUA_EUDORA Forged mail pretending to be from Eudora 0.0 CLICK_BELOW Asks you to click below The original message was not completely plain text, and may be unsafe to open with some email clients ; in particular, it may contain a virus, or confirm that your address can receive spam. If you wish to view it, it may be safer to save it to a file and open it with an editor.  ------------=_3FAE8656.371BED4D  Content-Type: message/rfc822; x-spam-type=original Content-Description: original message before SpamAssassin Content-Disposition: attachment Content-Transfer-Encoding: 8bit  Received: (qmail 25515 invoked from network); 7 Nov 2003 18:26:02 -0600 Received: from mailin-2.cc.uic.edu (HELO mailin-2.priv.cc.uic.edu) (128.248.155.213) by email0.cc.uic.edu with SMTP; 7 Nov 2003 18:26:02 -0600 Received: from mail3.mailnara.net (c-24-98-136-187.atl.client2.attbi.com [24.98.136.187]) by mailin-2.priv.cc.uic.edu (8.12.10/8.12.9) with ESMTP id hA80PxJk011669; Fri, 7 Nov 2003 18:26:00 -0600 Message-ID: <000016646728$00007347$[e-mail]> To: <[e-mail]> From: "brianj" <[e-mail]> Subject: Live your dream life!! MPNWSTU Date: Fri, 07 Nov 2003 15:32:41 -0800 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------=_1068251164-2528-687" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: QUALCOMM Windows Eudora Version 5.1 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.3018.1300 Content-Length: 2290 Lines: 72 Do you owe large sums of money? Are you stuck with high interest ra{tes? We can help! You can do what tens of thousands of americans have done, consolidate your high interest bills into one easy, low interest, monthly payment. By first reducing, and then completely removing your d+ebts, you will be able to start fresh. Why keep dealing with the stress, headaches , and wasted money, when you can consolidate your d+ebt and pay them off much sooner! Click below to learn more: http://61.186.254.9?affiliateid=mailer10 hjeuubnfs ------------=_3FAE8656.371BED4D-- 
 007347 
 From [e-mail] Fri Nov 7 18:26:05 2003 Received: from localhost [127.0.0.1] by localhost with SpamAssassin (2.60 1.212-2003-09-23-exp); Sun, 09 Nov 2003 12:24:22 -0600 From: "brianj" <[e-mail]> To: <[e-mail]> Subject: Live your dream life!! MPNWSTU Date: Fri, 07 Nov 2003 15:32:41 -0800 Message-Id: <000016646728$00007347$[e-mail]>  X-Spam-Status: Yes, hits=12.9 required=5.0 tests=CLICK_BELOW, FORGED_MUA_EUDORA,FROM_ENDS_IN_NUMS,MISSING_OUTLOOK_NAME, MSGID_OUTLOOK_INVALID,MSGID_SPAM_ZEROES,NORMAL_HTTP_TO_IP, SUBJ_HAS_SPACES,SUBJ_HAS_UNIQ_ID autolearn=no version=2.60 X-Spam-Flag: YES X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) X-Spam-Level: ************  MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------=_3FAE8656.371BED4D" This is a multi-part message in MIME format. ------------=_3FAE8656.371BED4D  Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 8bit Spam detection software, running on the system has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or block similar future email. If you have any questions, see the administrator of that system for details. Content preview: Do you owe large sums of money? Are you stuck with high interest ra{tes? We can help! You can do what tens of thousands of americans have done, consolidate your high interest bills into one easy, low interest, monthly payment. [...] Content analysis details: (12.9 points, 5.0 required) pts rule name description ---- ---------------------- ---------------------------------------------- 1.0 SUBJ_HAS_SPACES Subject contains lots of white space 4.3 MSGID_SPAM_ZEROES Spam tool Message-Id: (12-zeroes variant) 0.9 FROM_ENDS_IN_NUMS From: ends in numbers 0.2 NORMAL_HTTP_TO_IP URI: Uses a dotted -decimal IP address in URL 0.2 SUBJ_HAS_UNIQ_ID Subject contains a unique ID 4.3 MSGID_OUTLOOK_INVALID Message-Id is fake (in Outlook Express format) 0.1 MISSING_OUTLOOK_NAME Message looks like Outlook, but isn't 1.9 FORGED_MUA_EUDORA Forged mail pretending to be from Eudora 0.0 CLICK_BELOW Asks you to click below The original message was not completely plain text, and may be unsafe to open with some email clients ; in particular, it may contain a virus, or confirm that your address can receive spam. If you wish to view it, it may be safer to save it to a file and open it with an editor.  ------------=_3FAE8656.371BED4D  Content-Type: message/rfc822; x-spam-type=original Content-Description: original message before SpamAssassin Content-Disposition: attachment Content-Transfer-Encoding: 8bit  Received: (qmail 25515 invoked from network); 7 Nov 2003 18:26:02 -0600 Received: from mailin-2.cc.uic.edu (HELO mailin-2.priv.cc.uic.edu) (128.248.155.213) by email0.cc.uic.edu with SMTP; 7 Nov 2003 18:26:02 -0600 Received: from mail3.mailnara.net (c-24-98-136-187.atl.client2.attbi.com [24.98.136.187]) by mailin-2.priv.cc.uic.edu (8.12.10/8.12.9) with ESMTP id hA80PxJk011669; Fri, 7 Nov 2003 18:26:00 -0600 Message-ID: <000016646728$00007347$[e-mail]> To: <[e-mail]> From: "brianj" <[e-mail]> Subject: Live your dream life!! MPNWSTU Date: Fri, 07 Nov 2003 15:32:41 -0800 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------=_1068251164-2528-687" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: QUALCOMM Windows Eudora Version 5.1 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.3018.1300 Content-Length: 2290 Lines: 72 Do you owe large sums of money? Are you stuck with high interest ra{tes? We can help! You can do what tens of thousands of americans have done, consolidate your high interest bills into one easy, low interest, monthly payment. By first reducing, and then completely removing your d+ebts, you will be able to start fresh. Why keep dealing with the stress, headaches , and wasted money, when you can consolidate your d+ebt and pay them off much sooner! Click below to learn more: http://61.186.254.9?affiliateid=mailer10 hjeuubnfs ------------=_3FAE8656.371BED4D-- 
 [e-mail]>  X-Spam-Status: Yes, hits=12.9 required=5.0 tests=CLICK_BELOW,         FORGED_MUA_EUDORA,FROM_ENDS_IN_NUMS,MISSING_OUTLOOK_NAME,         MSGID_OUTLOOK_INVALID,MSGID_SPAM_ZEROES,NORMAL_HTTP_TO_IP,         SUBJ_HAS_SPACES,SUBJ_HAS_UNIQ_ID autolearn=no version=2.60 X-Spam-Flag: YES X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) X-Spam-Level: ************  MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------=_3FAE8656.371BED4D" This is a multi-part message in MIME format. ------------=_3FAE8656.371BED4D  Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 8bit Spam detection software, running on the system has identified this incoming email as possible spam.  The original message has been attached to this so you can view it (if it isn't spam) or block similar future email.  If you have any questions, see the administrator of that system for details. Content preview:  Do you owe large sums of money? Are you stuck with high   interest ra{tes? We can help! You can do what tens of thousands of   americans have done, consolidate your high interest bills into one   easy, low interest, monthly payment. [...] Content analysis details:   (12.9 points, 5.0 required)  pts rule name              description ---- ---------------------- ----------------------------------------------  1.0 SUBJ_HAS_SPACES        Subject contains lots of white space  4.3 MSGID_SPAM_ZEROES      Spam tool Message-Id: (12-zeroes variant)  0.9 FROM_ENDS_IN_NUMS      From: ends in numbers  0.2 NORMAL_HTTP_TO_IP      URI: Uses a dotted-decimal IP address in URL  0.2 SUBJ_HAS_UNIQ_ID       Subject contains a unique ID  4.3 MSGID_OUTLOOK_INVALID  Message-Id is fake (in Outlook Express format)  0.1 MISSING_OUTLOOK_NAME   Message looks like Outlook, but isn't  1.9 FORGED_MUA_EUDORA      Forged mail pretending to be from Eudora  0.0 CLICK_BELOW            Asks you to click below The original message was not completely plain text, and may be unsafe to open with some email clients; in particular, it may contain a virus, or confirm that your address can receive spam.  If you wish to view it, it may be safer to save it to a file and open it with an editor.  ------------=_3FAE8656.371BED4D  Content-Type: message/rfc822; x-spam-type=original Content-Description: original message before SpamAssassin Content-Disposition: attachment Content-Transfer-Encoding: 8bit  Received: (qmail 25515 invoked from network); 7 Nov 2003 18:26:02 -0600 Received: from mailin-2.cc.uic.edu (HELO mailin-2.priv.cc.uic.edu) (128.248.155.213)   by email0.cc.uic.edu with SMTP; 7 Nov 2003 18:26:02 -0600 Received: from mail3.mailnara.net (c-24-98-136-187.atl.client2.attbi.com [24.98.136.187])         by mailin-2.priv.cc.uic.edu (8.12.10/8.12.9) with ESMTP id hA80PxJk011669;         Fri, 7 Nov 2003 18:26:00 -0600 Message-ID: <000016646728 
 From [e-mail] Fri Nov 7 18:26:05 2003 Received: from localhost [127.0.0.1] by localhost with SpamAssassin (2.60 1.212-2003-09-23-exp); Sun, 09 Nov 2003 12:24:22 -0600 From: "brianj" <[e-mail]> To: <[e-mail]> Subject: Live your dream life!! MPNWSTU Date: Fri, 07 Nov 2003 15:32:41 -0800 Message-Id: <000016646728$00007347$[e-mail]>  X-Spam-Status: Yes, hits=12.9 required=5.0 tests=CLICK_BELOW, FORGED_MUA_EUDORA,FROM_ENDS_IN_NUMS,MISSING_OUTLOOK_NAME, MSGID_OUTLOOK_INVALID,MSGID_SPAM_ZEROES,NORMAL_HTTP_TO_IP, SUBJ_HAS_SPACES,SUBJ_HAS_UNIQ_ID autolearn=no version=2.60 X-Spam-Flag: YES X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) X-Spam-Level: ************  MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------=_3FAE8656.371BED4D" This is a multi-part message in MIME format. ------------=_3FAE8656.371BED4D  Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 8bit Spam detection software, running on the system has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or block similar future email. If you have any questions, see the administrator of that system for details. Content preview: Do you owe large sums of money? Are you stuck with high interest ra{tes? We can help! You can do what tens of thousands of americans have done, consolidate your high interest bills into one easy, low interest, monthly payment. [...] Content analysis details: (12.9 points, 5.0 required) pts rule name description ---- ---------------------- ---------------------------------------------- 1.0 SUBJ_HAS_SPACES Subject contains lots of white space 4.3 MSGID_SPAM_ZEROES Spam tool Message-Id: (12-zeroes variant) 0.9 FROM_ENDS_IN_NUMS From: ends in numbers 0.2 NORMAL_HTTP_TO_IP URI: Uses a dotted -decimal IP address in URL 0.2 SUBJ_HAS_UNIQ_ID Subject contains a unique ID 4.3 MSGID_OUTLOOK_INVALID Message-Id is fake (in Outlook Express format) 0.1 MISSING_OUTLOOK_NAME Message looks like Outlook, but isn't 1.9 FORGED_MUA_EUDORA Forged mail pretending to be from Eudora 0.0 CLICK_BELOW Asks you to click below The original message was not completely plain text, and may be unsafe to open with some email clients ; in particular, it may contain a virus, or confirm that your address can receive spam. If you wish to view it, it may be safer to save it to a file and open it with an editor.  ------------=_3FAE8656.371BED4D  Content-Type: message/rfc822; x-spam-type=original Content-Description: original message before SpamAssassin Content-Disposition: attachment Content-Transfer-Encoding: 8bit  Received: (qmail 25515 invoked from network); 7 Nov 2003 18:26:02 -0600 Received: from mailin-2.cc.uic.edu (HELO mailin-2.priv.cc.uic.edu) (128.248.155.213) by email0.cc.uic.edu with SMTP; 7 Nov 2003 18:26:02 -0600 Received: from mail3.mailnara.net (c-24-98-136-187.atl.client2.attbi.com [24.98.136.187]) by mailin-2.priv.cc.uic.edu (8.12.10/8.12.9) with ESMTP id hA80PxJk011669; Fri, 7 Nov 2003 18:26:00 -0600 Message-ID: <000016646728$00007347$[e-mail]> To: <[e-mail]> From: "brianj" <[e-mail]> Subject: Live your dream life!! MPNWSTU Date: Fri, 07 Nov 2003 15:32:41 -0800 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------=_1068251164-2528-687" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: QUALCOMM Windows Eudora Version 5.1 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.3018.1300 Content-Length: 2290 Lines: 72 Do you owe large sums of money? Are you stuck with high interest ra{tes? We can help! You can do what tens of thousands of americans have done, consolidate your high interest bills into one easy, low interest, monthly payment. By first reducing, and then completely removing your d+ebts, you will be able to start fresh. Why keep dealing with the stress, headaches , and wasted money, when you can consolidate your d+ebt and pay them off much sooner! Click below to learn more: http://61.186.254.9?affiliateid=mailer10 hjeuubnfs ------------=_3FAE8656.371BED4D-- 
 007347 
 From [e-mail] Fri Nov 7 18:26:05 2003 Received: from localhost [127.0.0.1] by localhost with SpamAssassin (2.60 1.212-2003-09-23-exp); Sun, 09 Nov 2003 12:24:22 -0600 From: "brianj" <[e-mail]> To: <[e-mail]> Subject: Live your dream life!! MPNWSTU Date: Fri, 07 Nov 2003 15:32:41 -0800 Message-Id: <000016646728$00007347$[e-mail]>  X-Spam-Status: Yes, hits=12.9 required=5.0 tests=CLICK_BELOW, FORGED_MUA_EUDORA,FROM_ENDS_IN_NUMS,MISSING_OUTLOOK_NAME, MSGID_OUTLOOK_INVALID,MSGID_SPAM_ZEROES,NORMAL_HTTP_TO_IP, SUBJ_HAS_SPACES,SUBJ_HAS_UNIQ_ID autolearn=no version=2.60 X-Spam-Flag: YES X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) X-Spam-Level: ************  MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------=_3FAE8656.371BED4D" This is a multi-part message in MIME format. ------------=_3FAE8656.371BED4D  Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 8bit Spam detection software, running on the system has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or block similar future email. If you have any questions, see the administrator of that system for details. Content preview: Do you owe large sums of money? Are you stuck with high interest ra{tes? We can help! You can do what tens of thousands of americans have done, consolidate your high interest bills into one easy, low interest, monthly payment. [...] Content analysis details: (12.9 points, 5.0 required) pts rule name description ---- ---------------------- ---------------------------------------------- 1.0 SUBJ_HAS_SPACES Subject contains lots of white space 4.3 MSGID_SPAM_ZEROES Spam tool Message-Id: (12-zeroes variant) 0.9 FROM_ENDS_IN_NUMS From: ends in numbers 0.2 NORMAL_HTTP_TO_IP URI: Uses a dotted -decimal IP address in URL 0.2 SUBJ_HAS_UNIQ_ID Subject contains a unique ID 4.3 MSGID_OUTLOOK_INVALID Message-Id is fake (in Outlook Express format) 0.1 MISSING_OUTLOOK_NAME Message looks like Outlook, but isn't 1.9 FORGED_MUA_EUDORA Forged mail pretending to be from Eudora 0.0 CLICK_BELOW Asks you to click below The original message was not completely plain text, and may be unsafe to open with some email clients ; in particular, it may contain a virus, or confirm that your address can receive spam. If you wish to view it, it may be safer to save it to a file and open it with an editor.  ------------=_3FAE8656.371BED4D  Content-Type: message/rfc822; x-spam-type=original Content-Description: original message before SpamAssassin Content-Disposition: attachment Content-Transfer-Encoding: 8bit  Received: (qmail 25515 invoked from network); 7 Nov 2003 18:26:02 -0600 Received: from mailin-2.cc.uic.edu (HELO mailin-2.priv.cc.uic.edu) (128.248.155.213) by email0.cc.uic.edu with SMTP; 7 Nov 2003 18:26:02 -0600 Received: from mail3.mailnara.net (c-24-98-136-187.atl.client2.attbi.com [24.98.136.187]) by mailin-2.priv.cc.uic.edu (8.12.10/8.12.9) with ESMTP id hA80PxJk011669; Fri, 7 Nov 2003 18:26:00 -0600 Message-ID: <000016646728$00007347$[e-mail]> To: <[e-mail]> From: "brianj" <[e-mail]> Subject: Live your dream life!! MPNWSTU Date: Fri, 07 Nov 2003 15:32:41 -0800 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------=_1068251164-2528-687" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: QUALCOMM Windows Eudora Version 5.1 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.3018.1300 Content-Length: 2290 Lines: 72 Do you owe large sums of money? Are you stuck with high interest ra{tes? We can help! You can do what tens of thousands of americans have done, consolidate your high interest bills into one easy, low interest, monthly payment. By first reducing, and then completely removing your d+ebts, you will be able to start fresh. Why keep dealing with the stress, headaches , and wasted money, when you can consolidate your d+ebt and pay them off much sooner! Click below to learn more: http://61.186.254.9?affiliateid=mailer10 hjeuubnfs ------------=_3FAE8656.371BED4D-- 
 [e-mail]> To: <[e-mail]> From: "brianj" <[e-mail]> Subject: Live your dream life!!                MPNWSTU Date: Fri, 07 Nov 2003 15:32:41 -0800 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------=_1068251164-2528-687" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: QUALCOMM Windows Eudora Version 5.1 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.3018.1300 Content-Length: 2290 Lines: 72 Do you owe large sums of money? Are you stuck with high interest ra{tes? We can help! You can do what tens of thousands of americans have done, consolidate your high interest bills into one easy, low interest, monthly payment. By first reducing, and then completely removing your d+ebts, you will be able to start fresh. Why keep dealing with the stress, headaches, and wasted money, when you can consolidate your d+ebt and pay them off much sooner! Click below to learn more: http://61.186.254.9?affiliateid=mailer10 hjeuubnfs ------------=_3FAE8656.371BED4D-- 

The SpamAssassin report is revealing . Despite the fact that this message includes several tricks to fool spam-checkers, such as random characters at the end and breaking up the words "rates" and "debt" with symbols, SpamAssassin identifies several suspicious characteristics and assigns a high spam score.