Although much of the information in this book is based on the Windows 2000 source code, you don't have to take everything on faith. Many details about the internals of Windows 2000 can be exposed and demonstrated by using a variety of available tools, such as those that come with Windows 2000, the Windows 2000 Support Tools, the Windows 2000 resource kits, and the Windows 2000 debugging tools. These tool packages are briefly described later in this section.
To encourage your exploration of Windows 2000 internals, we've included "Experiment" sidebars throughout the book that describe steps you can take to examine a particular aspect of Windows 2000 internal behavior. (You already saw one of these sections earlier in this chapter.) We encourage you to try these experiments so that you can see in action many of the internals topics described in this book.
In addition, this book comes with a CD-ROM that contains the latest version of the tools from www.sysinternals.com (a popular site for 32-bit Windows internals-related tools and information), as well as tools available only with this book.
Table 1-2 shows a list of the tools used in this book and where they come from. Although the capabilities of many of these tools overlap quite a bit in terms of the information that they can display, each of them shows at least one unique piece of information not available in any other utility.
Table 1-2 Tools for Viewing Windows 2000 Internals
|Dependency Walker||DEPENDS||Support Tools, Platform SDK|
|Dump Check||DUMPCHK||Support Tools, debugging tools, Platform SDK, Windows 2000 DDK|
|EFS Information Dumper||EFSDUMP||www.sysinternals.com*|
|Get SID tool||GETSID||Resource kits|
|Global Flags||GFLAGS||Support Tools, Platform SDK, Windows 2000 DDK|
|Handle/DLL Viewer||HANDLEEX, NTHANDLE||www.sysinternals.com|
|Kernel||I386KD , WINDBG, KD||Debugging tools, Platform SDK, debuggers Windows 2000 DDK|
|Object Viewer||WINOBJ||Platform SDK, www.sysinternals.com|
|Open Handles||OH||Resource kits|
|Page Fault Monitor||PFMON||Resource kits, Platform SDK|
|Performance tool||PERFMON||Windows 2000|
|Pool Monitor||POOLMON||Support Tools, Windows 2000 DDK|
|Process Statistics||PSTAT||Platform SDK, www.reskit.com|
|Process Viewer||PVIEWER (in Support Tools, Platform SDK the Support Tools) or PVIEW (in the Platform SDK)||Support Tools, Platform SDK|
|Quick Slice||QSLICE||Resource kits|
|Service Control tool||SC||Resource kits|
|Task (Process) List||TLIST||Support Tools|
|Task Manager||TASKMAN||Windows 2000|
* All tools from www.sysinternals.com are also included on the companion CD.
The companion CD contains the following unique tools that will assist you in exploring the internals of Windows 2000:
Many of the experiments throughout this book use the kernel debugger because it can easily display many internal Windows 2000 data structures and other details not available from any user-mode utility. Therefore, LiveKd will make trying these experiments much easier because it allows the kernel debugger to be used on a live system without requiring a second computer.
Whereas LiveKd displays internal kernel variables, the kernel variable performance counter extension DLL monitors the values of these variables over time. For example, these variables can contain numeric values of interest that might not be accessible through any of the Windows 2000 performance counters.
For more information about these tools, see the documentation provided on the CD as part of the tools installation. As a reminder, only people who buy this book can install and use these tools. They can't be further distributed. (See the license agreement in the back of this book for details.)
We'll refer to the Performance tool found in the Administrative Tools folder on the Start menu (or via Control Panel) throughout this book. The Performance tool has three functions: system monitoring, viewing performance counter logs, and setting alerts. For simplicity, when we refer to the Performance tool, we are referring to the System Monitor function within the tool.
The Performance tool can provide more information about how your system is operating than any other single utility. It includes hundreds of counters for various objects. For each major topic described in this book, a table of the relevant Windows 2000 performance counters is included.
The Performance tool contains a brief description for each counter. To see the descriptions, select a counter in the Add Counter window and click the Explain button. Or open the Performance Counter Reference help file in the resource kit. For information on how to interpret these counters to detect bottlenecks or plan capacity, see the section "Performance Monitoring" in the Windows 2000 Server Operations Guide, which is part of the Windows 2000 Server Resource Kit. These chapters provide an excellent description to anyone seriously interested in understanding Windows 2000 performance.
Note that all the Windows 2000 performance counters are accessible programmatically. The section "HKEY_PERFORMANCE_DATA" in Chapter 5 has a brief description of the components involved in retrieving performance counters through the Win32 API.
The Windows 2000 Support Tools consist of about 40 tools useful in administering and troubleshooting Windows 2000 systems. Many of these tools were formerly part of the Windows NT 4 resource kits.
You can install the Support Tools by running Setup.exe in the \Support\Tools folder on any Windows 2000 product distribution CD. (That is, the Support Tools are the same on Windows 2000 Professional, Server, and Advanced Server.)
The Windows 2000 resource kits supplement the Support Tools, adding some 200 additional tools. Besides including many tools useful for displaying internal system state, they contain useful internals documentation, such as the Registry Reference and Performance Counters help files.
There are two editions of the resource kits: the Windows 2000 Professional Resource Kit and the Windows 2000 Server Resource Kit. Although the latter kit is a superset of the former and can be installed on Windows 2000 Professional systems, none of the experiments in this book use the tools that are included only with the Windows 2000 Server Resource Kit. Be sure you visit www.reskit.com for updates to tools as well as for new tools.
Kernel debuggers are tools that device driver developers use to debug their drivers and support personnel use to troubleshoot hung systems and examine crash dumps (a copy of system memory saved in a file that can be analyzed to try and determine the cause of the system crash). Although a kernel debugger is used mainly for analyzing crash dumps or debugging device drivers, it is also a useful tool for investigating Windows 2000 internals because it can display internal Windows 2000 system information not visible through any standard utility. For example, it can dump internal data structures such as thread blocks, process blocks, page tables, I/O, and pool structures. Throughout this book, the relevant kernel debugger commands and output are included as they apply to each topic being discussed.
There are two versions of the Microsoft kernel debuggers: a command-line version (I386kd.exe for x86 systems*) and a graphical user interface (GUI) version (Windbg.exe). There is also a new version, Kd.exe, that replaces both of these. These tools are part of the debugger tools package, which is shipped in three places:
New versions of the debugging tools package are released independently of new versions of Windows 2000. Hence, you should occasionally check the Download section of Microsoft's Web site for the latest versions of these three packages. The debugging tools include a package called the OEM Support Tools, which is updated independently of the debugging tools and therefore might have newer versions of debugging-related tools (such as Kdex2x86.dll, a kernel debugger extension DLL with additional debugging commands).
The debugging tools help file, provided with each of the three packages just mentioned, explains how to set up and use the kernel debuggers (as well as other debugging and support tools that are part of the package). Additional details on using the kernel debuggers that are aimed primarily at device driver writers can be found in the Windows 2000 DDK documentation. There are also several useful Knowledge Base articles on the kernel debugger. Search for "debugref" in the Windows 2000 Knowledge Base (an online database of technical articles) on support.microsoft.com.
The kernel debugger has two modes of operation:
Detailed setup instructions can be found in the debugging tools documentation referred to previously.
The companion CD contains a tool called LiveKd that allows the use of the standard Microsoft kernel debugger on a live system, without needing two computers. LiveKd can be used for most of the experiments in the book and thus will be a helpful tool in exploring Windows 2000 internals.
You run LiveKd just as you would I386kd, Windbg, or Kd. LiveKd passes any command-line options you specify through to the debugger you select. By default, LiveKd runs the new command-line kernel debugger (Kd). If Kd is not found in the current directory, LiveKd tries I386kd. To run the GUI debugger (Windbg) specify the -w switch. To see the help on the switches for LiveKd, specify the -? switch.
LiveKd presents a simulated crash dump file to the debugger, so you can perform any operations in LiveKd that are supported on a crash dump. Because LiveKd is relying on physical memory to back the simulated dump, the kernel debugger might run into situations in which data structures are in the middle of being changed by the system and are inconsistent. Each time the debugger is launched, it gets a snapshot of system state, so if you want to refresh the snapshot, quit the debugger (with the "q" command) and LiveKd will ask you whether you want to start it again. If the debugger gets in a loop in printing output, press Ctrl+C to interrupt the output, quit, and rerun it. If it hangs, press Ctrl+Break, which will terminate the debugger process and ask you whether you want to run the debugger again.
Another debugging tool that doesn't require two machines for live kernel debugging is a third-party kernel debugger called SoftICE, which you can buy from Compuware NuMega. (See www.numega.com for details.)
To use any of the kernel debugging tools listed previously to examine internal Windows 2000 data structures (such as the process list, thread blocks, loaded driver list, memory usage information, and so on), you must have the correct symbol files for at least the kernel image, Ntoskrnl.exe. (The section "Architecture Overview" in Chapter 2 explains more about this file). The symbols are part of the Customer Support Diagnostics package (which, as mentioned earlier, is downloadable from www.microsoft.com). They are installed separately from the debugging tools and by default reside in the \Winnt\Symbols folder.
Later in the book, you'll see how you can use these symbol table files to display the names of internal Windows 2000 system routines and global variables.
Symbol table files must match the version of the image they were taken from. For example, if you install a Windows 2000 Service Pack, you must obtain the matching, updated symbol files for at least the kernel image; otherwise, you'll get a checksum error when you try to load them with the kernel debugger. These updated symbol files are not typically included or installed when you download and install a Service Pack from www.microsoft.com—they must be downloaded separately. (If you receive MSDN Professional or TechNet, they're included on the Service Pack CD-ROMs.)
The Platform SDK is part of the MSDN Professional (and Universal) subscription and can also be downloaded for free from msdn.microsoft.com. It contains the C header files and libraries necessary to compile and link Win32 applications. (Although Microsoft Visual C++ comes with a copy of these header files, the versions contained in the Platform SDK always match the latest version of the Windows operating systems, whereas the version that comes with Visual C++ might be an older version that was current when Visual C++ was released.) From an internals perspective, items of interest in the Platform SDK include the Win32 API header files (\Program Files\Microsoft Platform SDK\Include) as well as several utilities (Pfmon.exe, Pstat.exe, Winobj.exe). Some of the tools in the Platform SDK also come with the resource kits. Finally, a few of these tools are also shipped as example source code in both the Platform SDK and the MSDN Library.
The Windows 2000 DDK is part of the MSDN Professional (and Universal) subscription, but it is also available for free download at www.microsoft.com/hwdev. Although the DDK is aimed at device driver developers, the DDK is an abundant source of Windows 2000 internals information. For example, the DDK documentation contains a comprehensive description of the Windows 2000 I/O system in both a tutorial and reference form, including the internal system routines and data structures used by device drivers.
Besides the documentation, the DDK contains header files that define key internal data structures and constants as well as interfaces to many internal system routines (in particular, Ntddk.h). These files are useful when exploring Windows 2000 internal data structures with the kernel debugger because although the general layout and content of these structures are shown in this book, detailed field-level descriptions (such as size and data types) are not. A number of these data structures (such as object dispatcher headers, wait blocks, events, mutants, semaphores, and so on) are, however, fully defined in the DDK. In addition, the !dso command in the kernel debugger displays the format of many internal Windows 2000 data structures that are not defined in the DDK header files.
Many of the experiments in this book use freeware tools that you can download from www.sysinternals.com. Mark Russinovich, coauthor of this book, wrote most of these tools. Copies of these tools are in the \Sysint directory on the companion CD. In addition, a complete copy of the Web site www.sysinternals.com is also on the CD. (Keep in mind that although the version of www.sysinternals.com on the companion CD has the latest versions of tools available when this book was published, it won't have any new tools or updates that have been added to the live site later.) Many of these utilities involve the installation and execution of kernel-mode device drivers and thus require administrator privileges.