Registry Structure

For a better understanding of the registry logical structure, let's compare it to the file system that exists on the disk. The registry contains keys, which are similar to folders, and values, which can be compared to the files stored on the disk. Registry keys are container objects that can contain both subkeys and values. Registry values contain the data (like the files). The top-level keys of this hierarchical structure are called root keys.

The naming convention used to name registry keys and values is also similar to the one used to name files and folders. For example, compare a typical folder path like D:\WORK\BHV and a typical registry path: HKEY_LOCAL_MACHINE SYSTEM\CurrentControlSet.

Windows 2000/XP registry comprises the following five root keys: HKEY_CLASSES_ROOT, HKEY_CURRENT_USER, HKEY_LOCAL_MACHINE, HKEY_USERS, and HKEY_CURRENT_CONFIG. This hierarchical registry structure is illustrated in Fig. 1.6.

Fig. 1.6: Hierarchical structure of the registry

All the names of the root keys begin with the HKEY_string, which indicates that this is a handle that can be used by a program. The handle represents the value used to uniquely identify the resource that can be accessed by a program. Table 1.1 lists the root keys that exist in Windows NT/2000/XP registries and provides a brief description.

Table 1.1: Registry Root Keys

Root key	Description
HKEY_LOCAL_MACHINE	Contains global hardware information and operating system data, including bus type, system memory, device drivers, and other information used during the system boot process. Information on this key is applicable to all users who log in to the local system. There are three top-level aliases for this registry key: HKEY_CLASSES_ROOT, HKEY_CURRENT_CONFIG and HKEY_DYN_DATA
HKEY_CLASSES_ROOT	This key contains information concerning filename associations, OLE (Object Linking and Embedding) information associated with COM objects, and file-class associations (this data is equivalent to the registry that existed in earlier Windows versions). Parameters contained in this key are equivalent to those stored under the HKEY_LOCAL_MACHINE\Software\Classes key.
	You can find more detailed information on the HKEY_CLASSES_ROOT key in the "OLE Programmer's Reference" manual included with the Windows Platform Software Development Kit (SDK)
HKEY_CURRENT_CONFIG	Contains configuration data of the current hardware profile. Hardware profiles are sets of modifications introduced into the standard configuration of services and devices set by the Software and System subkeys of the HKEY_LOCAL_MACHINE root key. The HKEY_CURRENT_CONFIG key contains only changed data.
	Moreover, the data contained in this key is also contained in the HKEY_LOCAL_MACHINE\System\CurrentControlSet\HardwareProfiles\Current key
HKEY_CURRENT_USER	Contains the user profile of the user who's currently logged in to the system, including environment variables, desktop settings, network settings, and application settings.
	This key is a reference to the HKEY_USERS\user_SID key, where user_SID is the Security ID of the user who's currently logged in to the system
HKEY_USERS	Contains all active user profiles, including HKEY_CURRENT_USER and the default user profile. Users who can access the server through the network don't have profiles under this key because their profiles are loaded remotely on their workstations.
	Windows NT/2000/XP requires that each user who logs on to the system have his own user account. The HKEY_USERS key contains the \.Default subkey which is used to create a user profile for the user logging in to the system for the first time. It also contains other subkeys associated to the Security ID assigned to the appropriate user

Note 

As noted earlier, the registry concept was originally introduced in the first Windows NT version—Windows NT 3.1. Windows 9x/ME registries were developed based on this first version. At present, however, there are significant differences between the registries that exist in Windows 9x/ME and Windows NT/2000/XP. The most important one is that the Windows 95/98/ME registry has one more top-level key—HKEY_DYN_DATA. This key contains system information that may need continuous updating. Windows 9x/ME stores this information without flashing it to the hard drive. The HKEY_DYN_DATA key was present in all the Windows 2000 beta versions, but any attempts to open the key resulted in error messages. At present, the Regedit.exe and Regedt32.exe utilities in Windows 2000 and Windows XP Registry Editor (Regedit.exe) don't display this key.

Registry data are parameters stored within registry keys. Each parameter has its name, data type, and value. The three parts of the registry entry are also stored in the following order:

All registry data types defined and used in Windows NT/2000/XP are listed in Table 1.2.

Table 1.2: Registry Data Types

Data type	Description
REG_BINARY	Binary data. Most hardware components use binary information. Registry editors display this information in hex
REG_DWORD	This data is represented as a 4-byte binary value (double word). Most services and device drivers use this data type. Registry editors display this data using binary, hex, or decimal formats
REG_EXPAND_SZ	Expandable data string. This text string contains a variable name, which can be substituted by a variable value when called by an application
REG_MULTI_SZ	Multi-string field. Normally, all values that actually represent lists of text strings have this data type. A NULL character is used as a separator
REG_SZ	Text string in user-friendly format. Normally, this data type is used for component descriptions
REG_DWORD_LITTLE_ENDIAN	32-bit number in little-endian format. Equivalent of the REG_DWORD data type.
	When using the little-endian format, the lowest bit ("little end") appears first when representing the value. For example, the AO2Bh hex value will be represented as 2BA0.
	The little-endian format is used in Intel processors
REG_DWORD_BIG_ENDIAN	32-bit number in big-endian format. In contrast to the little-endian format, the highest bit ("big end") appears first when representing the value
REG_LINK	Unicode symbolic link. This data type is intended for internal use only.
	The REG_LINK is especially interesting because it allows one registry entry to reference another registry key or registry value. For example, if a registry contains the entry \Root1\Link with the REG_LINK data type and its value is \Root2\RegKey, and the RegKey key contains the RegValue value, then this value can be identified using the following two paths: \Root1\Link\RegValue and \Root2\RegKey\RegValue. Windows NT/2000/XP makes active use of this method. For example, some of the root keys listed in Table 1.1 are links to the nested keys of other root keys
REG_NONE	No defined data type
REG_QWORD	64-bit value
REG_QWORD_LITTLE_ENDIAN	64-bit value represented using little-endian notation. Equivalent of the REG_QWORD data type
REG_RESOURCE_LIST	List of hardware resources, which are only used in HKEY_LOCAL_MACHINE\HARDWARE
REG_FULL_RESOURCE_DESCRIPTOR	Hardware resource handle, which is only used in HKEY_LOCAL_MACHINE\HARDWARE
REG_RESOUECE_REQUIREMENTS_LIST	List of hardware resources, which is only used in HKEY_LOCAL_MACHINE\HARDWARE

Registry Data Storage

Windows NT/2000/XP stores registry entries as an atomic structure. The registry is subdivided into components, called hives for their resemblance to the cellular structure of a beehive. The Registry hive is a discrete body of keys, subkeys, and values rooted at the top level of the registry hierarchy. The main difference between registry hives and other groups of registry keys is that hives are constant registry components. Hives aren't created dynamically when the system boots, and aren't deleted when someone shuts the system down. Thus, the HKEY_LOCAL_MACHINE\Hardware key, which is created dynamically by the hardware recognizer when the system boots, can't be considered a hive.

Note 

As with Windows 2000, Windows XP registry also resides on the disk as multiple files called hives, serving as a repository for system configuration data. The registry code is redesigned for Windows XP, providing enhanced performance while remaining transparent to applications by using existing registry programming interfaces. Windows XP registry enhancements are mainly intended to provide performance improvements, and will be covered in more detail later in this chapter.

Registry hive data is stored in disk files, which in turn is stored in %SystemRoot%\System32\Config and %SystemRoot%\Profiles\Username folders (Windows NT 4.0). Windows 2000 stores registry hives in %SystemRoot%\System32\Config and %SystemDrive%\Documents and Settings\Username folders.

Each registry hive is associated with a set of standard supporting files. Table 1.3 lists standard Windows NT/2000/XP registry hives together with supporting files.

Table 1.3: Windows NT/2000 Registry Hives

Registry hive	Supporting files
HKEY_LOCAL_MACHINE\SAM	Sam, Sam.log, Sam.sav
HKEY_LOCAL_MACHINE\Security	Security, Security.log, Security.sav
HKEY_LOCAL_MACHINE\Software	Software, Software.log, Software.sav
HKEY_LOCAL_MACHINE\System	System, System.alt[*], System.log, System.sav
HKEY_CORRENT_CONFIG	System, System.alt[*], System.log, System.sav
HKEY_USERS\.DEFAULT	Default, Default.log, Default.sav
(Files that aren't associated with keys)	Userdiff, Userdiff.log, Userdifr[**], Userdifr.log[**]
HKEY_CURRENT_USER	Ntuser.dat, Ntuser.dat.log
[*]Files that were eliminated in Windows XP [**]Files that were first introduced in Windows XP

Note 

As you can see from this table, some registry files were eliminated in Windows XP, while other files were first introduced. This is due to the enhancements to the Windows XP registry, which will be covered in more detail later in this book.

All hive files, except for HKEY_CURRENT_USER, are stored in the %SystemRoot%\System32\Config folder.

The HKEY_CURRENT_USER hive is supported by Ntuser.dat and Ntuser.dat.log files. Ntuser.dat files contain user profiles, while Ntuser.dat.log files reflect all changes introduced to Ntuser.dat file. Windows NT 4.0 stores these files in subfolders of the %SystemRoot%\Profiles folder (except for the \All Users subfolder). Windows 2000 stores these files in %SystemDrive%\Documents and Settings %Username% folders.

The Ntuser and Userdiff files were first introduced in Windows NT 4.0:

• Ntuser.dat. This is the file where the user profile is stored. It replaced the usernamexxx and adminxxx files used in earlier versions of Windows NT.

• The Ntuser.dat file stored in the %SystemRoot%\Profiles\Default User folder replaced the Userdef file used in earlier versions of Windows NT. This user profile is used to create the HKEY_CURRENT_USER hive when the new user logs in to the system for the first time.

• Userdiff files stored in the %SystemRoot%\System32\Config folder aren't associated with any hive. They're used for updating user profiles that existed in previous Windows NT versions to make them compatible with Windows NT/2000 and Windows XP.

There are four types of files associated with registry hives. All of these file types, with the appropriate filename extensions, are listed in Table 1.4.

Table 1.4: Types of Files Associated to Windows NT 4.0/Windows 2000/Windows XP Registry Hives

Filename extension	Description
None	Contains the registry hive copy
ALT[*]	In Windows NT/2000, the ALT files contain the backup copy of the HKEY_LOCAL_MACHINE\System hive. The System hive is the only hive that has this type of backup copy stored in the System.alt file. As was already mentioned, ALT files were eliminated in Windows XP because registry code was redesigned to provide improved algorithms for faster queries, improved reliability, and larger registries
LOG	Contains the transaction log; any changes introduced to the keys and values make up this hive
SAV	Contains copies of registry hive files from the time the text-mode part of installation process was accomplished. There are SAV files for the following registry hives: \Software, \System, \SAM, \Security, and \Default.
	Windows NT/2000/XP makes backup copies of the registry hives during the installation process. The installation procedure consists of two parts; namely, the text-mode and the GUI-mode part of the installation. When the text-mode part of installation procedure is complete, the Setup program backs up the registry hives to the SAV files. This is done to protect the hives from failures which may occur during the GUI-mode part of setup. If such a failure occurs, the GUI-mode setup will resume after reboot, and SAV files will be used for rebuilding the registry hives
[*]Files that were excluded from Windows XP

Hive Atomicity and Recovery

The registry ensures the atomicity of individual operations. This means that any modifications done to a registry value (resetting, deleting, or saving) either work or don't work. This mechanism eliminates corrupt combinations of old and new registry values, in case the system stops unexpectedly due to a power failure, a hardware malfunction, or problems with the software. Consider, for example, a case when an application sets a value, and the system shuts down unexpectedly while this change is being made. After rebooting the system, this registry entry will be either reset to its previous value or have a new value, but no meaningless combinations of both registry parameters will appear. Moreover, the size and time data for the key containing the affected value will be accurate, whether this value has been changed or not.

Flushing Data

Windows NT 4.0 and Windows 2000 save registry data only after a flush occurs. Data flush occurs only after the modified data ages past a few seconds. Furthermore, data flush can be initiated by a direct call from the application, which intentionally flushes its data to the hard disk.

The system performs the following flushing procedure for all registry hives (except for the \System hive):

1. All changed data is saved to the LOG file of the respective hive, together with information on the exact location of the changed data within a hive. When this is done, the system performs a flush, and all modified data is written to the LOG file.

2. The first sector of the hive file is marked, which means that this file is in a transitional state.

3. The modified data is written to the hive file.

4. Finally, this hive file is marked as completed.

Note 

If the system shuts down unexpectedly when performing this procedure (between steps 2 and 4), it will recover the affected hive. When this hive is loaded during system startup (except for user profile hives, which are loaded when the user logs in to the system), the system will note the marker left in step 2, and it will continue recovering the hive using the changed data saved to the LOG file. Thus, if the hive isn't in transition, its LOG files aren't used. If the hive was in transition when the system stopped, it can't be loaded without a respective LOG file.

The \System hive is vitally important and is used at the earliest stages of the system boot. This hive includes the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control key, which contains information necessary for initializing the registry during the boot process. For example, registry entries stored with the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\hivelist specify the location of all other registry hives. Thus, if the \System hive is missing or corrupt, it can't be recovered using the procedure described above. Because of this, the system uses a different flushing process for the hive.

The System.alt file contains a copy of the System hive data. During the flush process, all changes are marked, saved, and then marked as completed. After this process is completed, the same flush process is done for the System.alt file. If there's a power failure, hardware malfunction, or problems with the software that cause system shutdown at any stage of this process, the system will try to find the correct information either in the System file or in the System.alt file.

The System.alt file is similar to the LOG file, except that the system during the boot will switch to using the System.alt file rather than trying to reapply all the changes saved in the LOG file. If the System file isn't marked as being in transition, then System.alt won't be needed.

Note 

Windows XP solves this problem by moving the registry out of paged pool and using the cache manager to do an in-house management of the registry. Therefore, there is no need to use the System.alt file in Windows XP.

Note 

Methods of storing registry data used by Windows 9x/ME and Windows NT/2000/XP also have significant differences. Windows 95/98 uses only two registry files, namely, System.dat and User.dat (in Windows ME there is another registry file, Classes.dat, which was introduced to improve boot time). To simplify the editing process, the registry editor displays both DAT files using unified interface. The User.dat file stores user-related information (for example, custom settings for the desktop), while the System.dat file contains system-related information, for example, default settings for the desktop, hardware profiles, network settings, and so forth. Both files have Hidden attribute and are stored in the Windows folder (the folder where the Windows 95/98 operating system is installed). Note that when installing Windows 95, the Setup program automatically creates backup copies for each of these files (User.da0 and System.da0, respectively). These backup copies are also hidden files stored in the Windows 95 directory. In contrast to Windows 95, there are no such files in Windows 98, because this operating system uses a newer and more efficient method of ensuring registry integrity. If the User.dat or System.dat files become corrupt, Windows 95 will make an attempt to use the User.da0 or System.da0 files. During the boot process, Windows 95 automatically updates these backup copies by replacing them with the current registry files. Theoretically, this mechanism ensures that the most up-to date configuration (the last best boot configuration) is always present on the computer.

Registry Size Problem

When we discussed the drawbacks of INI files earlier in this chapter, we mentioned the problems caused by the Win.ini size limitation. The Windows NT/2000 registry seemed to solve this problem, but it still remained limited in size. At the same time, there was a steady trend among registry consumers to use the registry like a database, which was constantly increasing the demands on registry size. The original design of the registry kept all of the registry files in the paged pool, which, in the 32-bit kernel, is effectively limited to approximately 160 MB because of the layout of the kernel virtual address space. A problem arose because, as larger registry consumers such as Terminal Services and COM appeared, a considerable amount of paged-pool was used for the registry alone, potentially leaving too little memory for other kernel-mode components.

Registry Size Limit in Windows NT/2000

To solve the above described problem, Windows NT 4.0/Windows 2000 provides the capability of restricting the size to which the registry can grow. To set the registry size limitation in Windows 2000, use the following procedure:

1. Double-click System in the Control Panel window. The System Properties window will appear.

2. Go to the Advanced tab and click the Performance Options button to open the Performance Options window (Fig. 1.7). Click the Change button in the Virtual Memory group of options. The Virtual Memory window will appear (Fig. 1.8). At the bottom of this window is the Registry size group with the Maximum registry size (MB) field, allowing the user to manually set the maximum registry size.

   
   Fig. 1.7: The Performance Options window

   
   Fig. 1.8: Setting the registry size limitation for Windows 2000

Windows NT/2000 registry data loaded to the memory is stored in a paged pool, which is a region in the physical memory used for storing data that can be flushed to the hard disk if these are not used for a long time. The size limit on the registry is set to prevent this sort of situation when the registry consumes all the space needed by other processes.

Registry Enhancements in Windows XP

In contrast to Windows NT/2000, in Windows XP the registry size limitation has been removed. The registry code is now redesigned to provide enhanced performance while remaining transparent to applications by using existing registry programming interfaces.

The new registry implementation delivers two key benefits:

• Larger registries

• Faster queries

Therefore, in Windows XP you won't find an option allowing you to establish a registry size limit (Fig. 1.9).

Fig. 1.9: In Windows XP, there is no option for setting the registry size limitation

Let us consider the above mentioned Windows XP registry enhancements in more detail.

• Larger Registries. Windows XP supports larger registries than previous versions of the kernel, which were effectively limited to about 80 percent of the total size of the paged pool. The new implementation is limited only by available system disk space. In Windows XP, this problem was solved by moving the registry out of the paged pool and using the cache manager to do the management of mapped views of the registry files. The mapped views are mapped in 256K chunks into system cache space instead of the paged pool.

• Faster Queries. Another issue that affected registry performance in earlier versions is the locality problem. Related cells are spread through all of the registry files. Accessing certain information, such as attributes of a key, could degenerate into page-faults, which lowers performance. The Windows XP registry uses an improved algorithm for allocating new cells that keeps related cells in closer proximity—such as keeping cells on the same page or nearby pages, which solves the locality problem and reduces the page faults incurred when accessing related cells. A new hive structure member tracks freed cells instead of relying on linked freed cells. When future cells are allocated, the freed cell list and a vicinity argument are used to ensure that they are put in the same bin as the hive.

Windows XP improves the way the registry handles big data. In versions before Windows XP, if an inefficient application steadily increased a value by a small increment, it created a sparse and wasteful registry file. Windows XP solves this problem with a big cell implementation, where cells larger than 16K are split into increments of 16K chunks. This reduces fragmentation when the data length of a value is increased within a certain threshold.