4.8 Personal folders and offline folder files

---

The Exchange Store is the repository for messages and other items created by users. The Store has many notable attributes: high performance, single place to check for incoming virus attacks, availability from any client, and so on. However, from a user perspective, the drawbacks of server-based storage are that each mailbox has a set quota enforced by the Store and that mailboxes are only available when the server is running and the network can connect clients to the server.

You can use personal folder (PST) files to provide additional storage to users, while offline folder files (OST files) enable people to work when a network link is unavailable or they cannot otherwise reach the server.

4.8.1 Personal folders

PSTs reside on either a PC drive or space allocated to users on a network share. You can even create a PST on a floppy disk, a feature that you can use to transfer small amounts of folder data from one PC to another. The concept behind PSTs is a "personal repository," differentiating storage from the server-based mailbox and public folders. PSTs allow users to create and manage their own private document archives. All servers have some storage limitations in terms of either the total available physical disk space or the mailbox quota allocated to individual users. It is easy to run out of space, especially if you are in the habit of creating or circulating messages with large attachments. In these situations, users can reduce their mailboxes under the quota by moving files to a PST.

Up to Outlook 2003, an ANSI-format PST can grow to a maximum of 2 GB, and the new unicode format allows PSTs to expand to fill disks, if you allow them to. The new default size limit is 20 GB (more than enough for even the most fastidious of human packrats), although administrators can limit this to even less than 2 GB by setting the registry keys described in Table 4.9 at:

HKEY_CURRENT_USER\Software\Policies\Microsoft\Office\ 11.0\Outlook\PST

Table 4.9: Registry Values to Control PST and OST File Sizes

Name	Value Range	Default Value	Meaning
MaxLargeFileSize	0x00000001-0x00005000	0x00005000 (20,480 in decimal or 20 GB)	Maximum size of a unicode PST or OST
WarnLargeFileSize	0x00000001-0x00005000	0x00004C00 (19,456 in decimal or 19 GB)	Size when Outlook issues warning that a PST or OST is becoming too large
MaxFileSize	0x001F4400-0x7C004400	0x7BB04400 (1.933 GB)	Maximum size for an ANSI-format PST or OST
WarnFileSize	0x00042400-0x7C004400	0x74404400 (1.816 GB)	Size when Outlook issues warning that a PST or OST is becoming too large

All values are DWORDs and expressed in megabytes (unicode format) or bytes (ANSI format). Administrators can control all these values through policies.

Clearly, there are many older Outlook clients in use, so a danger exists that users could attempt to store more than 2 GB in a PST and corrupt the file. Microsoft released preventative fixes in Office XP SP1 and service release 1A for Office 2000. The Office XP fix issues a warning when a PST gets to 1.8 GB, while the Office 2000 fix just stops the file from growing. If you get into a situation where a PST grows past the supported limit, you can use a Microsoft utility called PST2GB^[5] to truncate the file. However, truncation will lose data, so it is best to keep PSTs small.

You cannot convert an existing PST file to unicode format, so if you want to use a PST larger than 2 GB, you have to create a new file and populate it from scratch, including dragging and dropping items from older PSTs. People often use PSTs to transfer documents between each other, so remember that older versions of Outlook cannot open unicode-format PSTs and use older format files whenever you need to swap files in this fashion.

Unless it is password-protected, no security credentials are required to open a PST, so you can browse a PC or net share, find a PST, and then open it with any MAPI client, even if it is not associated with your own mailbox. You can set a password on the PST when you create the file (Figure 4.31) or by changing the properties of the PST afterward. You must then provide the password before the client can open the file. Microsoft's official policy is that there is no way that they can recover data from a PST if the user forgets his or her password. However, this is a legal rather than a technical issue and you can easily break into a PST through utilities such as PS19UPG, which circulate freely around the Internet. Of course, if an accident happens and you corrupt the PST by using an unsupported utility, you are on your own and Microsoft support will be very unsympathetic.

Figure 4.31: Properties of a PST.

Encouraging users to work with PSTs is fine as long as they use the same PC all the time and understand their responsibilities to manage and protect data. However, if users move from one PC to another they will leave information behind them. In these situations, server-based storage (which is accessible from any PC, and from any client) is a better solution. One way to accommodate roaming users is to place their PSTs in their directories on a networked file server. If you do this, remember to tell users to always fully log out from each PC they use; otherwise, they will not be able to connect to their PST when they move to another PC. Microsoft did not design the internal structure of a PST to support multiple concurrent accesses.

4.8.2 Mail delivery to personal folders

You can configure Outlook to deliver new messages to a PST through the Delivery tab of the client Tools | Options menu option (Outlook 98 or 2000) or by selecting the PST file as the preferred mail delivery location, as shown in Figure 4.32 (Outlook 2002 and later). Rushing to divert messages from your mailbox to a PST is not a good idea, and you should consider a few points before you make what is, to some extent, a radical change in a basic function of the messaging system. When you select the option to deliver messages to a PST, the messages remain in the server mailbox until you connect to the server. Outlook then transfers the messages from the server Inbox folder to an Inbox folder in the PST. After the messages transfer, Exchange deletes them from the mailbox. Some users do not expect this to happen and are very surprised when they look at their server-based Inbox to find that all their mail has apparently disappeared! Of course, the messages are available in the PST, but users often overlook this fact, leading to any number of frantic calls to the help desk to demand the return of the missing messages.

Figure 4.32: Selecting personal folders as the mail delivery location.

Delivering mail to a PST is similar in concept to the traditional POP3/ IMAP4 email model provided by ISPs. This model holds some attractions if you want to build Exchange servers to host very large numbers of mailboxes with a restricted amount of storage. The economics behind hotmail.com would be very different if each user had mailbox quotas of 50 or 100 MB, common allocations in corporate messaging systems.

If server-based mailboxes are used, the space available to each user is constrained by his or her mailbox quota. If PSTs are used, the server's storage is largely transient, because Exchange only holds messages on the server until users connect to fetch messages. Users can therefore be allocated relatively small quotas within the Store (10 MB or less), allowing many more users to be supported on a single server. Delivering messages to PSTs or using PSTs as secondary storage was much more common in the early days of Exchange, but cheaper hardware and software that is more functional have reduced the attractiveness of these options. Given the relatively low cost of disk storage and the ability to partition the Store, most Exchange designers now choose to deploy server-based storage and have downgraded the importance and need for PSTs (in this respect).

Additionally, multiple client access has become increasingly important. Exchange 4.0 offered a single client and single access protocol (MAPI). Exchange now supports MAPI, IMAP, POP3, and Web clients, but only MAPI clients can use PSTs. If you opt to deliver mail to a PST, then you immediately restrict your options for client deployment, so you lose much of the value of being able to make an occasional connection to your mailbox with Outlook Web Access, since only new messages are kept there.

Diverting incoming email to a PST also eliminates the effect of the shared message model, which is not quite as important now as it was in the past.

The arrival of email-enabled viruses creates yet another factor for administrators to consider. There are many server-based antivirus agents, all of which are able to protect thousands of mailboxes from a single point. Microsoft has utilities to scan a complete mailbox store to find and disinfect viruses, but no utilities exist to scan multiple PSTs for viruses. If a virus arrives and passes by an outdated virus detection pattern and is intended for delivery to hundreds of PSTs, you will have a huge amount of work to track down each instance of the virus and eliminate it.

It is a fact that you can only read a message delivered to a PST if you have physical access to the PST. In other words, an administrator who can access the contents of server-based mailboxes cannot see the messages in the PST unless he or she can access them. The only messages that remain in the mailbox are those that have arrived since the last time the user connected and picked up mail.

In cases where the PSTs used for mail delivery are located on networked file services, the responsibility for backing up the PSTs remains with the system administrators. Backing up individual PSTs is easier than backing up a very large mailbox store, chiefly because there is a wider range of backup software available that is capable of processing individual files (the PSTs). Restoring an individual file from a backup tape is much easier than restoring a user mailbox from a backup of the Store, and this fact alone makes the use of PSTs something to consider if you do not run a backup program that supports bricked backups and restores (and, indeed, uses the facility).

While on this point, it is interesting to note that MAPI clients automatically disconnect from a PST after 30 minutes of inactivity, allowing system administrators to plan to take file-level backups of PSTs stored on a network share even if users leave their PCs logged on (hopefully protected by a password-enabled screen saver). The next time a client operation commences that uses an item in the PST, the client automatically reestablishes the connection.

Table 4.10 lists the pros and cons of using PSTs.

Table 4.10: Pros and Cons of Using PSTs

Pros	Cons
A PST can be stored on any available device, including floppy disks.	You cannot share a PST between users. For example, you cannot use a PST to share documents between a manager and a secretary. Users who leave a PC logged on can inadvertently block their own access to a PST if they move to another PC.
PSTs can be used to transfer files between users, including users in different Exchange organizations.	You can encrypt a PST to protect its content against other users, but if the password is lost, there is no supported way to recover it. However, unsupported utilities exist that can crack PST passwords.
An "old format" PST can expand to any size up to a theoretical 2-GB limit, subject to available disk space. The largest PST in use is well over 1 GB (there is a limit of 64,000 items in a single folder). A unicode-format PST can grow much larger than 2 GB. Thus, PSTs allow users to hold more information, so they become a more attractive personal archive.	Delivering mail to a PST removes the benefit of single-instance storage (within a single mailbox store). In addition, unless you use Outlook 2003, double the space is required to store a message, because the PST holds items in both RTF and ASCII format.
You can back up PSTs with any file-level backup utility.	If you use PSTs, make sure that someone is actually backing up the files on a regular basis. There is an increased chance of message corruption, because there is no rollback/transaction log mechanism available for PSTs, so you may need to go back to a backup to recover data.
PSTs are easier to restore than a complete mailbox store.	You cannot send deferred mail messages unless you connect the client to the server.
The Auto-Archive feature in the Outlook client uses PSTs to store archived items.	Inbox Assistant rules are not processed unless the client is connected to the server.
PSTs provide an easy way for Microsoft Mail users to move to Exchange, because they can convert MMF files to PSTs.	There is a decreased ability for system administrators to monitor disk use, because messages and other items can be stored on the server and scattered across user PSTs.
The EXMERGE utility uses PSTs to move mailbox contents between servers.	Extra network traffic is generated to move information to and from PSTs. Messages for delivery to a PST are held on the server until a client connects, and if a lot of mail is waiting the initial connect can take a long time.
	You can only access PSTs from MAPI clients.
	The ESM "Move Mailbox" option does not move items stored in PSTs. If you move users to a new server, you may also have to move their PSTs to a suitable local file share.
	It's difficult to come up with a good strategy to clean items infected by a virus if they are delivered to a PST.
	PSTs may not be available to roving users-for example, people who want to work with the same data at home and in the office.

Some installations encourage users to view their server mailbox as a repository for working documents, a place that holds information that must be available no matter which client they use to connect to the server. According to this model, users are supposed to move items to a PST once the files are no longer current or become less important. The model assumes a fair amount of discipline on the part of users, something that is not always easy to find. It also devolves responsibility for data security to users after data transfers to user control. One of the undoubted advantages of server-based storage is the assurance that someone else will take care of backing up all your important data. People are not that good when the time comes to create backups or copies of local data on a PC.

4.8.3 Configuring PSTs

Users are not limited to connecting to a single PST, and you can configure multiple PSTs for use, perhaps having one file on a networked file service and another on the local PC disk, with maybe a third on a floppy disk. Putting a PST on a floppy disk may seem strange, but it is certainly a valid method to transfer files from one PC to another or from one user to another. You must configure each PST individually in the MAPI profile. By default, the display name of each PST is "Personal Folders," so things can get slightly confusing if you do not change the name afterward to reflect the intended use of the PST. For example, use names such as "Network Personal Folders," "Archived Folders," "Mail Folders," and so on. You can easily change the display name of a PST through Outlook by selecting the PST and then bringing up its properties, as shown in Figure 4.33. Administrators are unable to configure or control PSTs through ESM. Remember: These are personal folders.

Figure 4.33: Changing the properties of a PST.

Users moving to Exchange from old LAN-based mail systems such as Microsoft Mail or Lotus cc:Mail often find PSTs a very natural extension of the mode of working they are used to. It is normal with LAN-based email systems for clients to download messages from a post office to a local mailbox file. With Exchange server, the opposite is true, since the recommended way of working is to leave messages on the server rather than moving them to the PC. Users can move messages to a PST if they wish, albeit through a drag-and-drop operation rather than having the mail system do it automatically. Installations moving from LAN-based email systems should carefully consider whether they want to migrate megabytes of user messages to Exchange. In my opinion, it is best to leave messages where they are-on the PC by converting the older format mailbox to a PST. Apart from reducing the demand for server resources, this approach means that users can be migrated to Exchange much more quickly.

Training sessions clearly offer an opportunity to encourage good work habits such as the effective use of PSTs. Trainers can use hint or cheat sheets to brief users on how you would like them to work and so relieve some pressure on server-based resources. As with most features, there are pros and cons that you need to understand before you can make the most effective use of them. Make sure that you understand the impact of these issues before you rush into using PSTs.

4.8.4 PST archiving

Users, despite the suspicions of many system administrators, are only human. As such, they will accumulate mail messages and other items in their mailboxes in the same way as they collect anything else. Some people will be very neat and tidy, maintaining an efficient and effective folder structure. Others will let chaos exist, with messages piling up in folders created for a purpose long forgotten. Most users exist between the two extremes. They do their best to keep their mailboxes reasonably clean, eliminating dead wood from time to time, but, generally, there is a lot of excess information hanging around mailboxes, all occupying valuable space on the server.

Archiving attempts to provide automatic clean-up assistance to users by allowing them to define criteria to control when to move items from the "live" (server-based) store to a "passive" (PST) store. Archiving obsolete email helps to increase overall system performance by reducing the number of items in folders and the overall size of the mailbox stores. The more items in a folder, the slower it is to access. This is especially so for folders such as the Inbox, Journal, and Calendar.

Archiving criteria are age based. In other words, messages become archival candidates after they exceed a certain age: 28 days, 15 weeks, or maybe 6 months. The net effect of moving messages from the live store is to reduce the amount of information held on the server, although you do not reduce the actual physical size of a mailbox store unless you rebuild the database with the ESEUTIL program. Of course, reducing the space occupied by databases is good from a system administrator's point of view. More space is available for new items, optimal use is made of disks, and you reduce backup times.

In Exchange, archiving is client driven and the server plays no great role, apart from processing client requests during the archival process. Outlook is the only client that includes archiving features, either on an automatic basis or on an explicit basis according to criteria defined by the user.

Outlook can archive messages in two ways. First, access the "Archive" option on the File menu to invoke the dialog shown in Figure 3.34 (Outlook 2002 running on Windows XP Professional). The default option reviews the contents of selected folders (and subfolders) and archives any items contained in the folders that are older than a specified date. The items are archived immediately into the archive file, which is a PST located on any available disk.

Figure 4.34: Archiving an Outlook folder.

Alternatively (by clicking the first radio button in the dialog), you can scan all folders in the mailbox and process items according to the autoarchive settings (Figure 4.35). Auto-archive settings can be set on an individual folder-by-folder basis, or you can apply the same criteria to all folders. Note that Outlook archives items based on a message's last modified date rather than the received or created date. The following actions change an item's modified date:

• Forwarding

• Replying

• Replying to all

• Editing and saving

• Moving or copying

Figure 4.35: Outlook 2002 auto-archive options.

The second option you can take is to auto-archive, which Outlook runs according to a schedule that you create (Figure 4.35). The default interval is 28 days. When Outlook starts, it checks to see if auto-archiving is enabled and then how long it is since it archived items. If archiving is enabled and the time interval has elapsed, Outlook prompts to determine whether it should begin to process folders.

Auto-archiving can take a long time. The initial implementation of archiving (prior to Outlook 98) suffered because moving items from server to a PST is a slow process. Users who hit the "OK" button when they were given the option to archive quickly learn that the process could take up to 30 minutes to complete (depending on the folders and the number of items in each folder), and during that time they could do nothing else. Current versions of Outlook perform archiving in a background thread, much like background synchronization. Users can continue to work while archiving proceeds, albeit with reduced response times.

^[5] . See http://www.microsoft.com/downloads/release.asp?releaseid=44353.

---