4.5 Junk mail processing

---

All versions of Outlook support rules to allow you to automate common tasks. In an Exchange environment, rules divide into two types: server-side rules and client-side rules. Exchange can execute server-side rules set on a mailbox without clients. For example, you do not need to log in for the Out of Office rule to send out of office notifications. Client-side rules normally require a user to authenticate by logging on to a mailbox and can only execute after the client connects to a mailbox and begins to download messages. Previous versions of Outlook attempted to suppress junk mail with rules, but the growing volume of spam and the more sophisticated techniques used by spammers to avoid detection mean that the rules-based approach is ineffective. The Outlook 2003 junk mail filter is brand new; it does not use the previous rules-based approach coupled with a static list of keywords to detect junk mail. Because the client executes the code to detect junk mail, messages remain in your Inbox until you connect Outlook, at which point processing begins. However, you can only use the junk mail processing feature if you configure Outlook 2003 in cached Exchange mode or connect to a server with POP3 (a protocol that always stores messages locally). Outlook could connect in the traditional manner and process messages online, but would need to fetch the message content from Exchange before the client could filter the messages. This approach works for small messages, but the network communication overhead required to fetch messages for checking is excessive, so Outlook limits this feature to messages stored in the local cache.

4.5.1 Detecting junk mail

Some junk mail is easy to detect. Messages that come from people in Nigeria who offer you incredible opportunities to earn millions of dollars if only you send them a couple of thousand to grease the wheels of commerce. Messages from young women who want to perform a range of personal services. Then there are the messages from those who have many interesting drugs to enhance your performance. The list goes on.

Spam detection software relies on a mixture of detection techniques to identify unwanted messages, including looking at originator addresses to block messages from well-known spammers (the blackhole lists) as well as message content to pick up phrases such as "Viagra" and "porn." Detection software also analyzes message structure. In the past, Outlook rules were not capable of performing the sophisticated filtering that anti-SPAM software can, but Outlook 2003's junk mail feature can suppress a very high percentage of the spam that creeps through corporate defenses and penetrates your Inbox. In this respect, you can view Outlook's junk mail filter as implementing another layer for defense in depth against spam, much as you run a desktop antivirus tool to supplement the antivirus software run on servers.

4.5.2 How Outlook's junk mail filter works

If you opt for some level of junk mail protection, Outlook begins to process new messages waiting in the Inbox as soon as it starts up and checks incoming messages as they arrive. If you do not want Outlook to look for junk mail, select the "No protection" level at the Junk E-Mail Options dialog reached from Tools.Options (Figure 4.18). The default protection level is Low, meaning that Outlook will only detect obvious spam. Based on the level of spam that arrives in my Inbox, I prefer to set the protection level to "High," meaning that Outlook aggressively checks for spam and moves any message that seems to be spam into the Junk Mail folder. Outlook automatically creates the Junk Mail folder if it does not already exist, and you do not have the option to select another folder. However, you can opt to delete junk mail immediately (the equivalent of using the shift-delete option to remove messages without going through the deleted items folder) instead. This is a recommended option only if you have a very high level of confidence in the filters that Outlook uses to catch spam.

Figure 4.18: Junk mail options.

You cannot change the algorithm Outlook uses to decide whether a message is spam, but you can help Outlook improve its level of accuracy by creating lists of safe senders and blocked senders. Safe senders are email addresses that you recognize and do not want Outlook to mark as spam. Blocked senders are the precise opposite and are addresses that you gather from spam that elude Outlook's filters. To mark a message as spam, right-click and then select the Junk E-mail option, then "Add Sender to Blocked Senders list." Outlook adds the sender's email address to its list of known spammers, as shown in Figure 4.19. You can add anyone you like to this list, including colleagues. However, Outlook rejects any attempt to add a sender from within your organization (those who appear in the GAL) to the junk senders list. This is a pity, because there are always a few individuals who generate mail that you really do not need to read.

Figure 4.19: Adding a message to the Blocked Senders list.

Outlook's junk mail filter works as follows:

• Checks email against your contacts and assumes that any message from a contact is safe to deliver.

• Checks email against the corporate GAL and assumes that you are willing to read messages from anyone in the GAL.

• Checks email against the user's "Safe Senders list" and passes any matching messages through.

• Checks email against the user's "Safe Recipients list" and passes any matching messages through.

• Checks email against the user's "Blocked Senders list" and transfers any matches to the Junk Mail folder.

• Runs the spam filter.

The filter generates a ranking (think of the ranking as being a number from 1 to 100) to determine whether a message seems, behaves, and feels like spam. The higher the number, the more spam-like Outlook believes a message to be. After a value is determined, Outlook decides whether to refile the message into junk mail. If you set a low protection level, messages with relatively high spam values will get through. If you set a high protection level, Outlook removes any message that even smells of spam. Sometimes, the high protection level is too aggressive and Outlook refiles legitimate messages into the Junk Mail folder. This is a good reason to check the messages in the Junk Mail folder from time to time before deleting them, just in case. If you find that Outlook is consistently picking out messages from specific correspondents, you can add them to your "Safe Senders list" to take care of the problem.

The use of the Blocked Senders and Safe Senders lists is apparent: The purpose of the Safe Recipients list is less so. Essentially, a safe recipient is a way to identify messages sent to a particular destination (often a distribution list or newsgroup) that you receive copies of and want to receive. You do not control the membership of the distribution list or newsgroup, but you assume that the administrator will make sure that no one uses the list for spam. Therefore, you want to tell Outlook that these messages are OK and you do this by identifying the email address of the list as a safe recipient.

After a while, you will accumulate a list of spammers that you may want to share with others. You may also want to share lists of safe recipients and safe senders. You can export or import data into any list. Figure 4.20 shows a list of people who sent me spam. Taking the "Export to file" option generates a simple text file that you can manipulate with any text editor. You can append lists gathered from different users and share updated lists of known spammers from a central location so that anyone can load them into Outlook. Note that you can also add complete domains to your Blocked Senders list to block any attempt to send you email from any of those domains. However, be careful not to be too enthusiastic about adding individuals or domains to the Blocked Senders list, since large lists will only slow down processing.

Figure 4.20: Blocked Senders list.

Even the best implementation of bastion servers to protect networks against incoming spam will let some messages through. There are just too many messages circulating to block everything, and the spammers come up with new tricks regularly that fool corporate defenses temporarily. If you take the time to capture details of spammers as junk mail arrives, gradually you improve Outlook's ability to recognize and block new spam. In my case, the junk mail filter intercepts most offending messages and I regularly capture a hundred or so junk messages each month. With previous versions of Outlook, rules can slow down delivery to a mailbox, especially when they call for complicated processing such as the type necessary to detect spam. Outlook 2003 caches the different lists it uses and implements the junk mail filter in compiled code, so performance is acceptable. By this, I mean that I perceived no great difference on mail delivery with the junk mail filter in place. Note that if you use cached Exchange mode, Outlook does not perform junk mail filtering until it has fully downloaded the header and content of new messages.

While it is good for Outlook 2003 to have its own junk mail block, the client leverages some Exchange server features too. Outlook stores the safe and blocked lists as well as its junk mail settings in user mailboxes to allow OWA to use the same data when it checks for junk email. Exchange can also exploit this information with antispam tools, which run on the server, and Exchange honors the Blocked Senders list to redirect email into the junk mail folder as soon as it arrives on the server without Outlook getting involved. Outlook 2003 and OWA also both block external content to prevent spammers from getting hold of valid email addresses; they use your Safe Senders list to ensure that you can see any content that you receive from sources that you know to be safe. See section 5.4 for more details.

If, like me, spammers generate intense annoyance and you want to take action against people who send you unwanted email, you can take out an account with an antispam company and use it to report spammers to the carriers that facilitate their email. It is important to use a "blind" email account to report spam, because spammers generally welcome responses, since they prove that their messages are getting through. In my case, I use a www.spamcop.net account. To report a message, you paste details of the message's path from its header into a form on the antispam site, which then analyzes the header data to determine whether the originator is a known spammer or otherwise exhibits the attributes of a potential spammer. You can then decide to send messages to the administrator of the domains that carried the message along its path to your domain. As shown in Figure 4.21, I used the Options menu when viewing a message to see the header information and then pasted it into spamcop.net, which reported that the message came through a site included on several lists of well-known spammers. Fighting spammers by protesting their activities through sites such as spamcop.net seems to reduce the amount of spam, possibly because these people do not like anyone to report their activities to ISPs that host their activities.

Figure 4.21: Checking a suspected Outlook message for spam.

Of course, Outlook's junk mail processing technology is not available to you if you do not have Outlook 2003. If you use an earlier version of Outlook and want to suppress spam, you have to consider third-party add-on products such as Sunbelt Software's IHateSpam (www.sunbeltsoftware.com), which supports Outlook 2000 and 2002. Deploying add-on products is not a popular option with system administrators because of the purchase cost, work to deploy theutility to desktops, and ongoing support, but it is certainly something you may have to look at if spam becomes a major problem. The alternative is to plan an early upgrade to Outlook 2003, an option that also incurs cost but at least you end up with the functionality that is part of the base Outlook product supported by Microsoft.

Figure 4.22: Creating a rule from a message.

4.5.3 More about rules

Outlook 2003 does not deliver much in terms of extra functionality for rules and you are still restricted to a 32-KB limit for the maximum size of server-side rules for a mailbox. Exchange lore suggests that this limit is because of a MAPI RPC restriction, but, in fact, it is due to the Store, which assumes that data passes in chunks of 32 KB or less. However, no equivalent limit exists for the number of client-side rules that Outlook supports. Outlook 2003 does allow users to create rules based on a message. The idea is that it is a lot easier for users to pick a message and say that they would like to process similar messages automatically. You can, therefore, select a message and then right-click and take the "Create Rule" option to begin creating a rule based on the message. As you can see in Figure 4.22, Outlook prepopulates a dialog with information extracted from the message. You can amend the details, like defining a folder for Outlook to refile the message into, or click on Advanced Options to work with the normal Rules Wizard.

---