7.14 ExIFS-the Exchange Installable File System

---

When Exchange 2000 shipped, it came with a new installable file system called ExIFS, or the Exchange Installable File System. Since then, people have spent a lot of time figuring out just how useful ExIFS is. Apart from the technical aspect of being able to navigate the contents of mailboxes and folders in the same way you can move within the NTFS directory structure, it is difficult to pinpoint exactly how ExIFS enhances Exchange.

Originally, Microsoft's grand plan for the second generation of Exchange, beginning with Exchange 2000, was to transform the Store from an application-specific repository to be an enterprise repository. This meant that you would be able to hold much more than simple email items and attachments in the Store and that many more clients would emerge to become consumers of the Store. Of course, the Store has been able to deal with any type of attachment since day zero, but the consumer side was limited to email clients. Over time, the client set grew, but never went past an email focus. At one time, some at Microsoft believed that public folders could be a candidate to replace network shares and tried to sell this idea to anyone who would listen. However, the grand plan foundered, because public folder access is limited to MAPI, IMAP, and browser clients, and most public folder implementations suffered from poor performance as soon as you stored more than a couple of hundred items in a folder. Other issues existed to limit the universal usefulness of the Store, but performance and client access are large problems to solve.

Another issue is the long-term direction for the Store. When Microsoft designed ExIFS, the Microsoft .NET initiative had not taken root and the concept of presenting data and application services based on common standards such as XML did not exist. Basing Store access on a rudimentary file access protocol such as SMB does not make sense in a .NET world, so there is not the same need or interest for something like ExIFS.

Hindsight is a wonderful way to look at anything. Given today's knowledge, it might not seem that Microsoft took a bold move in Exchange 2000 when it introduced ExIFS in an attempt to open up the Store, but at the time, ExIFS was an interesting and worthwhile advancement. Microsoft reasoned that if it engineered support for the SMB protocol into the Store, then any application that used SMB to access a file system (normally FAT or NTFS) would be able to use the Store. These applications might want to access messages, attachments, and other items held in the Store, but, more attractively, developers might make the decision to use the Store as the preferred repository for application-specific data. Another attraction for application developers is that they can use ExIFS to access the Store without generating RPCs. Applications such as Web browsers and the Internet Protocol virtual servers (POP, IMAP, etc.) that Exchange uses deal with files using standard NT file operations. The presence of ExIFS allowed these servers to continue with the normal approach of passing buffers, sockets, and file handles to what seemed to be a normal file system, whereas Exchange hides behind the screen. Apart from anything else, performance is better if you manipulate files like this instead of via RPCs, because Windows has no user-mode processing to perform.

ExIFS works by mapping the Store and representing it in a form that applications supporting SMB can understand. In Exchange 2000, this means that applications see drive M: (you can change the drive letter if required) as the entry point to the Store. If the application has sufficient permission, it can navigate down through drive M: to access the contents of mailboxes and public folders.

HTTP-DAV presented another issue for ExIFS. Getting to a file is one thing; getting to a file and all the metadata that surrounds it to make it a mail message or attachment is another. HTTP-DAV allows programmers to avoid MAPI RPCs and is more in line with the .NET direction, so the combination of performance and access to more data means that it is hard to argue for ExIFS as a general-purpose programming interface.

ExIFS still survives in Exchange 2003, albeit in a more restricted and background role. Most of the reasons for this positioning are that the experience gained with ExIFS since the launch of Exchange 2000 has not been good. Sure, applications can certainly access the contents of the Store through ExIFS, but instead of using this ability in a good and productive manner, applications tended to use ExIFS as a way around the protection that Exchange normally gives to the Store when you access it through a client such as Outlook or an API such as CDO or ADO. Well-behaved applications respect permissions, but examples exist where applications have used ExIFS to try to work around permissions and break security, attempt to take backups, or even use ExIFS as a well-intended route to protect mailboxes against viruses. Backups are a specific instance, where the file-level- type access available to backup utilities via ExIFS is enough to destroy all of the contextual information that Exchange needs to know, including where data fits into stores, mailboxes, and public folders. You can certainly take a backup through ExIFS, but you have little chance of being able to reassemble all the pieces from the resulting save set if you ever need to restore. Some people have tried, all have failed, and all have had a hugely frustrating time when they discovered just how important contextual information is.

The net result from these experiences is that Microsoft now views ExIFS as merely an internal file system that Exchange can use for its own purposes to perform optimal streaming and I/O activities such as data transfers across process boundaries. The Store also uses ExIFS in some of its interactions with the STM file.

The net result of this experience is that Microsoft took the decision to hide the ExIFS drive from view in Exchange 2003, and you have to reveal it if you decide that you have some applications that could use ExIFS. In most cases, it is best to leave well enough alone and let ExIFS sink back into the mists of hidden processing performed on an Exchange server.

7.14.1 The role of ExIFS

Even though ExIFS no longer has the future originally mapped out by Microsoft, it is still interesting to explore the technology because it is still used by Exchange 2000 and Exchange 2003. ExIFS allows Win32-based programs to map mailboxes and public folders in the Store as if they were just another network drive, and this fact is leveraged extensively by the virtual protocol servers hosted in the INETINFO (IIS) process to perform file-type data manipulation on Store data with minimal overhead.

Figure 7.49 shows how ExIFS links Win32 programs to the Store. Common Win32 file manipulation functions (CreateFileEx, ReadFileEx, Find- FileFirst, etc.) called by programs such as Windows Explorer that want to use Store data are directed to a logical device that is managed by a kernel- mode driver called EXIFS.SYS, which also handles programmable access to content via the FileSystemObject object. Note that programs access the STM file using ExIFS, because its storage is file oriented rather than the traditional database page-style layout used in the EDB. The role of ExIFS is to intercept and redirect the function calls made by the applications to the Store. As far as programs such as Windows Explorer are concerned, they make calls to retrieve and process data stored on a DOS device. In essence, ExIFS makes the contents of the Store appear to Win32 applications as a collection of folders and individual files in exactly the manner they would expect to see when browsing any other DOS device. ExIFS also supports all the file system-related functions exported from KERNEL32.DLL.

Figure 7.49: ExIFS architecture.

Figure 7.50 shows ExIFS in action. In this case, I have logged on to my mailbox with Outlook Express and can scroll through the items in the Sent Items folder. In the background, Explorer browses the same items using its normal interface. Outlook Express connects with IMAP; Explorer sees the mailbox content via ExIFS.

Figure 7.50: Viewing the same items through Windows Explorer and Outlook Express.

The right-hand Explorer pane shows the "MBX" folder. This is the logical root for all mailboxes on the Exchange server and individual mailboxes show up as folders under MBX. Standard Windows permissions prevent unauthorized access to a mailbox via ExIFS. All the mailboxes are visible (in the same way as you can see other users' mailboxes in the GAL), but any attempt to access a folder in someone else's mailbox is firmly denied. However, unlike the GAL, ExIFS lists folders for hidden mailboxes.

After you install Exchange 2000, users can map to their own mailboxes by using the M: drive as a network share. M: is the default drive letter allocated by ExIFS when it loads, connects to the Store, and initializes the file system. If a server already has allocated drive M: for another purpose, ExIFS selects the next available letter. Apart from the fact that Microsoft Mail used an M: drive and that "M" stands for "Mail," there is no great architectural reason why Microsoft selected M: as the default letter for ExIFS. Probably each one of the engineers involved in the design has his or her own version of why this particular letter was chosen. In practical terms, M: avoids the most common letters allocated on servers (the selection of C:, D:, E:, F:, G:, and H: probably cover the letters allocated to volumes on 80 percent of all servers), but Microsoft could equally have selected another letter toward the other end of the scale (such as X:). However, administrators often allocate letters toward the end of the alphabet to network shares and the easiest way to avoid conflict is to select a letter in the middle.

ExIFS creates a folder for a user under the M: drive root the first time an item appears in his or her mailbox. If users wish, they can map another DOS drive to their own mailbox or public folder, and, indeed, issue a NET USE command to make a folder available to other users as if it were just another network share. There is no doubt that being able to map a mailbox like a DOS drive is very cool, and it does hammer home the message of "access from anywhere" in an effective manner. Taking our analogy of a database as file system a little further, the picture is complete now because ExIFS allows you to view the content of messages and other items within the Store represented as individual files arranged within a file system. The biggest advantage delivered by ExIFS is that you do not have to install any additional software on a client system before you can use ExIFS as long as the standard network redirector is available.

The Store and IIS must be operational before you can access the contents of the M: drive. If either of these components is not running, the M: drive is unavailable to applications and users. Thus, if you have an application that depends on the M: drive (perhaps to create a document in a particular folder) and access the drive through a network share, you have to make sure that you share out the drive each time Exchange 2000 is started. However, it is not a good idea to use ExIFS as the basis for any application access, because there is no guarantee that Microsoft will continue to support ExIFS in future releases of Exchange.

Unlike Exchange 2000, both SharePoint Portal Server and Exchange 2003 hide the ExIFS drive. You must expose the ExIFS drive temporarily or permanently to access its contents from the command prompt or Windows Explorer. In addition, if you run Exchange 2000, you can go the reverse route and hide the ExIFS drive by following the instructions in Microsoft Knowledge Base article 305145.

To expose ExIFS temporarily, use the SUBST command from a command prompt to map the special backofficestorage mechanism, as follows:

C:> SUBST M: \\.\backofficestorage

Exchange 2000 uses drive M: by default, so it should be available without the need to make a change to the server. You can select any other unused letter if you prefer, although it is a good idea to have a common approach across all servers. You have to update the registry to change the drive letter used by ExIFS or on an Exchange 2003 server if you want to expose the ExIFS drive permanently. Do not reenable the ExIFS drive unless you have a good reason to do so. To enable the drive or change the existing allocated letter, go to the key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\ EXIFS\Parameters 

Then select the "DriveLetter" value or add it if it does not exist, and type "M" or whatever drive letter you chose into the value. Figure 7.50 shows the SUBST command and the same data viewed from Outlook Express and Windows Explorer. In this case, we are looking at the Sent Items folder. In Windows Explorer, the items have a .EML extension and a message icon, while Outlook Express displays them as normal.

Figure 7.51 illustrates the point. In this case, we access the contents of an Exchange folder via DOS. In DOS terms, the folder path is the Windows domain name followed by the "MBX" root, mailbox alias, and then folder name. Once mapped, you can use DOS commands to manipulate the items and folders within the mailbox. For example, you can add a new file by copying it into the mailbox from another directory on another drive. To demonstrate this point in action, Figure 7.52 shows the result when I used the TYPE command to display the contents of an email, which Exchange identifies by giving the item an .eml extension when it generates its DOS file name.

Figure 7.51: Listing an Exchange mailbox folder with the DOS DIR command.

From: "Tony Redmond" [e-mail] To: [e-mail] Subject: Access to MSDN and TechNet Date: Fri, 6 Dec 2002 17:52:17 -0000 MIME-Version: 1.0 Content-Type: multipart/alternative;      boundary="----=_NextPart_000_0023_01C29D50.37DC1570" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.3663.0 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3663.0     This is a multi-part message in MIME format.     ------=_NextPart_000_0023_01C29D50.37DC1570 Content-Type: text/plain;      charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable     Folks,     With reference to yesterday's message about the new arrangement for HP = employee access to MSDN and TechNet, the question was asked if you would = have electronic access to the Microsoft MSDN site. We have checked this = and confirm that all registered employees will have access. = = 20 ------=_NextPart_000_0023_01C29D50.37DC1570 Content-Type: text/html;      charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable     <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML><HEAD> <META http-equiv=3DContent-Type content=3D"text/html; = charset=3Diso-8859-1"> <META content=3D"MSHTML 6.00.3663.0" name=3DGENERATOR> <STYLE></STYLE> </HEAD> <BODY bgColor=3D#ffffff> <DIV><FONT face=3DArial size=3D2>Folks,</FONT></DIV> <DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV> <DIV><FONT face=3DArial size=3D2>With reference to yesterday's message = about the new=20 arrangement for HP employee access to MSDN and TechNet, the question was = asked=20 if you would have electronic access to the Microsoft MSDN site. We have = checked=20 this and confirm that all registered employees will have access.= 20     </LI></UL></SPAN></DIV></BODY></HTML>     ------=_NextPart_000_0023_01C29D50.37DC1570----

Figure 7.52: Example output from DOS TYPE command (Outlook Express 6.0).

While it might seem obvious, it is important again to emphasize that ExIFS deals with Store data just like files, so a lot of metadata goes missing when you use ExIFS. Some message metadata is always present, because it is part of the information held alongside message content in the STM file. Essentially, this is the data specified by RFC 822, such as message subject, recipients, originator, and so on, so this explains what you see when you use TYPE to view the message. When Exchange fetches message data with ExIFS, it also fetches the metadata from the EDB file through the epoxy layer and ends up with a complete message. DOS-like programs such as TYPE have no knowledge about epoxy, metadata, or anything to do with messaging, so they ignore this data. The Store performs the work to combine STM data with EDB data and with information such as priority, flags, attachments, and so on and presents the message body part in the correct format (plain text, RTF, or HTML). Clients do not need to be aware that these complexities exist and can concentrate on the user interface.

The output listed in Figure 7.52 comes from a short note sent to a distribution list. If you view this output through a DOS command box, the content scrolls continually to the screen until it reaches the end and is impossible to follow, so it is better to redirect output to a text file to examine the message structure. In this case, Outlook 2000 generated the message in RTF format (it could also have been generated in HTML) and stored it in a mailbox. When ExIFS receives the request from DOS to provide content, it retrieves the message and converts it into a readable format.

For many good reasons, including the experience Microsoft gained of ExIFS in action and the advent of Microsoft .NET, while a good foundation was laid by enabling file system access to Store data in Exchange 2000, Microsoft decided not to develop the technology further in Exchange 2003, so we are left with an incomplete work. For example, ExIFS does not support all possible Win32 calls. Enough is there to allow programs to insert and retrieve messages and attachments from the Store exactly as they request information held on an NTFS or FAT drive. As a demo, the technology works and proves the point that SMB-compliant applications can map and access Store data, but Microsoft did not understand the ramifications of the access well enough.

In the examples, where we use the DOS TYPE command to examine the content of a message, you see raw content without formatting or reference to any of the properties that clients merge with content to represent the totality of any item in the Store. No support is available for DOS-level access to calendar, tasks, or journal items. You can see the file names of these items, but any attempt to access a file results in the message "The system cannot find the file specified."

Because of its flaws and shortcomings, it is best if you leave ExIFS alone to its internal function, where Exchange can take care of all of the interaction between EDB and STM, Store and IIS, client and server. ExIFS is an interesting demonstration, showing that technology can bridge the gap between data and wildly different representations (DOS and Outlook, for instance), but it is not a prime time component any more. HTTP-DAV is often the best solution if you want programmatic access to Exchange data.

---