7.4 The streaming file

---

In daily life, you most commonly encounter streamed data in the form of audio or video files. Streamed files are normally very large, so clients access them in a continuous stream. Even a DVD video, which allows much more selective sampling of a film than is normally available in a standard video format, breaks up a film into scenes, each of which may be hundreds of megabytes in size. Audio files are no different, especially if they are recorded at a high sample rate to achieve the best possible quality. If you use the "Record narration" feature of PowerPoint to record notes for a presentation, you will be surprised at just how quickly your hard disk fills.

Even if we do not want to encourage users to send massive attachments to each other, the simple fact is that they do. Every holiday, a new variation on the electronic greeting card is made available somewhere on the Web, and it ends up being mailed millions of times to different users. In the days of "green screen email," we had cards composed of video escape character sequences. The escape characters instructed the terminal to display a series of primitive graphic characters in the correct order to create a picture. With a lot of dedication and trial and error, people produced amazing effects, and the electronic card was born.

The internal structure of EDB databases is not very suitable for storing large attachments that clients access in a continual stream. Think of how the Store needs to divide a 10-MB attachment containing the latest electronic greeting card into 2,500 separate pages inside the database. No guarantee is given that these pages will be contiguous to each other; so reading the data requires that the Store perform substantial processing. Internally, the streaming file is organized into clusters similar to an NTFS on-disk structure. This structure is more suited to the fast streaming of information than if you had to retrieve data from multiple points within a file, which is the case with the EDB. Originally, Microsoft referred to the STM as an SLV database, because it is designed to hold "super-long-value" content, which gives you an insight into the intention behind its introduction.

Table 7.5 shows the split in responsibilities between the streaming file and the EDB databases. The streaming file holds native content generated by Internet clients such as Outlook Express or Web browsers, while MAPI clients such as Outlook ignore the streaming file and continue to use mail- box or Public Stores as before. The Store handles all the necessary processing to hide the interaction between the two databases when MAPI clients need to retrieve content stored in the streaming file by Internet clients. The streaming file only stores content to enable fast access to data that is not suitable for storage in a Mailbox or Public Store. In terms of processing, it is costly to retrieve attachments such as the 25-MB Star Wars trailer when the Store has to fetch data from multiple individual pages that are unlikely to be stored contiguously. Of course, you can stop users from sending such large attachments by setting message limits on connectors, as we will see in Chapter 8.

Table 7.5: Differences between the EDB and Streaming File

EDB Database	Streaming File
Holds content (HTML or RTF) generated by MAPI clients	Holds content generated by Internet clients and manages data created for the ExIFS
Holds indexed message properties (author, date sent, etc.) generated by all clients. The Store promotes properties automatically for items held in the streaming file into the EDB database as they are accessed.	Does not hold any message properties
Holds attachments generated by MAPI clients (native format)	Holds attachments generated by Internet clients (MIME format)
Access organized in 4-KB pages	Access via data streaming in 4-KB pages

The streamed database solves this issue by allowing clients to access data in a continuous stream. The properties of all messages, such as the subject, author, and recipients are stored in a Mailbox or Public Store, where ESE indexes item properties automatically to make them available for searching. The indexing referred to here is different from full-text indexing, which is an optional feature and is controlled by a property set on each database.

---