The Core of the Messaging System

The exploration of the post office directory structure earlier in this chapter briefly discussed the relationships between the various files. This section covers these relationships in more detail.

The User Database

Every user in the GroupWise system is assigned to a unique user database per post office. These files are found in the OFUSER directory. They follow the naming convention USERxxx.DB, in which xxx is replaced with the three-character file ID for the user.

Note

Customers who have migrated from WordPerfect Office might on occasion have users with a two-character file ID.

This file ID, or FID, is alphanumeric, which means that you can have up to 46,656 uniquely named user databases on each GroupWise post office. The FID must be unique at the post office level, but no such restrictions exist system-wide. Users are also assigned a globally unique ID, or GUID, to ensure uniqueness system-wide, but this string is far too long to be conveniently used as part of a filename.

Contents of a User Database

The user database contains the following types of records:

• Pointers

• Personal items

• Folders

• Personal address books

• Preferences

These five record types are explained in the sections that follow.

Pointers

When you open a user mailbox with a GroupWise client, you will see a list of folders, and in each folder you will see different items. These are simply database records. Each item record contains enough data to make the display meaningful, but the item itself usually exists in a different database. The item records found in the user database serve as pointers to the items in message databases. There are also special pointers, called document references, that point to records in GroupWise libraries.

Personal Items

Some items are completely contained in the user database, however. These posted or personal items do not have recipients or senders. Personal notes, tasks, and appointments, as well as posted items, fall into this category.

Folders

All the items in a user mailbox can be stored in several folders, and this folder structure is also stored in the user database. This means that whenever a user database needs to be re-created, all nonpersonal items can be restored again from the message database, but these items will be in the system Cabinet folder without the earlier folder structure.

Personal Address Books

Addresses that users enter and groups that users create exist in the personal address books. One such book is the frequent contacts list, which is populated automatically with the addresses of individuals with whom you exchange email.

Preferences

Many user settings and preferences are kept in the user database. These include rules, send options, categories, button bars, proxies, signature options, and the user password. Other settings, such as window positions, are kept in the Windows Registry.

Message Database Content

Prior to GroupWise 7, on every GroupWise post office, you would find up to 25 message databases. These files follow the naming convention MSGn.DB, in which n is a number from 0 to 24. Message databases contain group itemsitems that have been sent from one user to other users.

With GroupWise 7, architectural changes have been made so that the system can make more than just 25 message databases. In fact, with GroupWise 7 there is now a potential of 255 message databases.

When a message is delivered to a user, that user gets a pointer in his or her user database. The pointer points to the actual message item record in a message database. When a user reads a subject line from the Mailbox folder, the text comes from the item record in the user database. When that user double-clicks the message to open it, the pointer is followed, and the resulting window is populated from the appropriate item record in a message database.

The User Database/Message Database Relationship

When a GroupWise user sends a message, the complete item record is first written to the sender's message database (which is shared with other users). Next, a pointer is written to the sender's user database. This shows up as a sent item. If the message was sent to a user in the same post office as the sender, the recipient's user database gets a pointer to the same message, which exists in the message database. If the recipient exists in a post office different from the sender's, when the message arrives at the destination, it is written first to the same message database as the sender's. The destination user's user database then receives a pointer to the local message database.

Every message a user sends will be written to the same message database. Each user is assigned to a message database for sending based on the user FID.

Item records in message databases contain all data pertaining to that item, including status informationwhen it was delivered, opened, or deleted, or when any other action was performed against it. If items are attached to a message, they will also be included as part of that item record, unless the item is larger than 2KB. Attachments, message bodies, or any other item fields that exceed 2048 bytes in length are written as separate records called binary large objects (BLOBs).

Binary Large Objects (BLOB Files)

In the OFFILES directory, there are up to 255 subdirectories. Each of these can contain binary large object files. These BLOB files are standalone records. They are referenced as records by pointers in user and message databases, and they contain any field data for item record fields that exceed 2048 bytes in length. Following are just a few examples of how BLOB files are created:

• With attachments: If you attach a 2MB file to a mail message, you end up creating a BLOB file in the OFFILES structure, as well as a message record in your assigned message database. If your attachment is less than 2KB, however (say, a 1000-byte autoexec.bat file), the attachment is encoded as a field in that message record, it exists entirely within your assigned message database, and no BLOB file is created.

• With long messages: If you type a 10-page message and send it, your mail item exists as a record in the message database you are assigned to, but the text of that message is now in a BLOB file somewhere in the OFFILES structure. This is simply because the length of your message exceeds the 2KB limit stored in each record of the message database.

• With multiple addressees: If you address a message to every user in your company, your TO: field will very likely exceed 2048 bytes in length. Thus, your message body record could exist in your assigned message database (if it's under 2KB), but the distribution list for that message will be in a BLOB file.

BLOB files are given hexadecimal names with alphanumeric extensions. They are encrypted using GroupWise's native 40-bit encryption, and they are compressed as well. BLOB files in the OFFILES structure cannot be read independently; they must be opened by following the appropriate item pointers via the GroupWise client.

The Purpose of the Deferred Message Database

In the OFMSG directory, there is a special-purpose database, used for delayed delivery messages and for special item lists used by the system when GroupWise mailboxes are moved between post offices. This file, named NGWDFR.DB, is called the deferred message database.

There are two operations a user can perform that will result in records being written to this database:

• Sending a message with Delay Delivery checked

• Sending a message with Expiration Date checked

You can check both of these options for individual items from the Send Options tab of a message you are composing. You can make these settings for all items by choosing Tools, Options, Send from the GroupWise client.

Delayed Delivery

When delayed messages are sent, GroupWise must write to four places:

• Message database: The message database, assigned to this user for send operations, gets the item record.

• User database (of the sender): The user database gets a pointer, as usual.

• Deferred message BLOB: The message in its entirety is kept in a special BLOB file. These BLOB files are kept in the OFFILES\FD0FDF directories.

• Deferred message database: NGWDFR.DB gets a pointer to that item record, along with a date indicating when the item should be delivered to its recipients.

The reason the message is placed in its entirety as a BLOB file is that the message should not show up in the recipient's mailbox until the delivery time. Entries in NGWDFR.DB tell the POA when to deliver a message, as well as which BLOB file corresponds to that message.

Messages with Expiration Dates

When a message is sent with an expiration date, it is sent normally. That is, records exist in the sender's user database, the recipient's user database, and the sender's assigned message database. An additional record is placed in NGWDFR.DB: a pointer to the message database record, along with the date on which that record should be deleted if it has not yet been opened by the recipient.

Tip

Although you can set expiration-date options globally, this is a very poor way to manage mailbox size. Expired items are deleted only if they are unread, and the NGWDFR.DB database file will grow to be quite large if it must keep track of every sent message.

How Prime User Databases Are Made

To explain the function of a prime user database, this section uses variables, like in high school algebra.

A prime variable is one that represents a new iteration of the original variable. A prime user database is a new iteration of the original user database, containing only a subset of the contents of that original database.

Now, if you are not mathematically inclined, that example might not work. Here's an approach to the prime user database in terms of functionality.

When a user shares a folder or an address book with a user on another post office, a prime user database will be created on that post office. This database will contain any shared folders or address books the user has shared with anyone on that post office.

The items that go into a shared folder are written either to the appropriate message database on the sharee's post office, or, in the case of personal and posted discussion items, directly to the prime user database.

Prime user databases cannot be named the same way user databases are. On large systems, the capability to create uniquely named databases would be compromised. Thus, they are named as PUnnnnnn.DB, in which nnnnnn is a six-digit number. This number is in two-byte hexadecimal pairs, beginning with 020101.

The first shared folder to arrive at a post office will arrive in PU020101.DB, regardless of which user shared the folder. The next user who shares a folder or an address book with that post office will end up creating PU020102.DB.

PU databases are created only for folders or address books that are shared across post offices. If they are shared with users on the same post office only, they will be accessed through the sharing user's USERxxx.DB file.

Using the Mailbox Index to Find Mailbox Items

Every item on the GroupWise system is full-text indexed. This allows the powerful Find feature of the GroupWise client to locate items in your mailbox, even if you have forgotten which folder you filed them in. This full-text index is created by the POA during the QuickFinder Indexing process.

There are actually two sets of indexes in a GroupWise post office. The mailbox index is found in OFUSER/INDEX. There is a library index as well, which is discussed later in this chapter.

In the OFUSER/INDEX directory, there are two types of files:

• *.IDX: The permanent index files. They are updated and renamed at the end of each day. There is typically one of these for each user mailbox.

• *.INC: The incremental index files. They are created during each QuickFinder Indexing run, and at the end of each day they are appended to the appropriate *.IDX files.

The names of these files are based on the time they were created or updated. The filenames are in hexadecimal format, so the file 000000F1.INC was created before the file 000000FF.INC.

Keeping Track of Databases with the Guardian Database

With all the various files floating around in a GroupWise post office directory, a central naming authority is required. The NGWGUARD.DB file, or the guardian database, is the central naming authority. This file contains a catalog of every database in the GroupWise information store on this post office. Any process that needs to work with a GroupWise database must first hook into the guardian database.

Because NGWGUARD.DB represents a potential single-point-of-failure, the GroupWise POA backs up this file each time it loads, as well as at midnight. Here's how the process works:

1.	The POA loads and checks the structure of NGWGUARD.DB. If it checks out okay, the process skips to step 5.
2.	The POA renames the bad NGWGUARD.DB to NGWGUARD.DBA.
3.	The POA copies the last backup, NGWGUARD.FBK, to NGWGUARD.DB.
4.	The POA writes all transactions from NGWGUARD.RFL (the guardian roll-forward log) to NGWGUARD.DB. The POA now has a clean NGWGUARD.DB, with all transactions made since the last backup rolled back into it.
5.	The POA copies NGWGUARD.DB to NGWGUARD.FBK and to OFMSG\GUARDBAK\NGWGUARD.FBK.
6.	Delete NGWGUARD.RFL and create a fresh, blank version of this file.
7.	When changes are made to NGWGUARD.DB, write these changes to NGWGUARD.RFL.

This is how the changes between guardian backups are stored for recovery.