Basics

Once you have the Debugging Tools for Windows installed, there's still some work to do before you can effectively use WinDBG and friends. In this section, I'll talk about getting WinDBG to use your Symbol Server, setting a few options to make your life a little easier, and dealing with the WinDBG window idiosyncrasies.

Symbol Server Setup

Back in Chapter 2's "Set Up a Symbol Store" section, I discussed the steps necessary to set up the two symbol-related environment variables, _NT_SYMBOL_PATH and _NT_EXECUTABLE_IMAGE_PATH, so you can get symbols loaded. In the "Debugging with Source Servers" section of Chapter 2, I showed how to set the _NT_SOURCE_PATH environment variable to enable the source server for WinDBG. I'm assuming that you've performed those steps to ensure symbol and source happiness. As I stated in that section, WinDBG doesn't give you much information about when it's encountering symbol load problems. In order to ensure that you have the best symbols possible, I will go through the exact steps to get the Symbol Server set up correctly, which can be done with a simple test. I'm going to take you through the steps here but not completely explain the commands you're executing until later in the chapter.

Testing the Symbol Settings

Once you've set the _NT_SYMBOL_PATH environment variable, make sure to start WinDBG either with Microsoft Windows Explorer or a new Command window started from Explorer so the environment variable gets picked up. Here are the exact steps to follow:

1.	Ensure you have a live connection to the Internet.
2.	Start WinDBG.
3.	On the File menu, select Open Executable.
4.	Navigate to your Windows directory and double-click Notepad.exe.
5.	If you get a "Save information for workspace?" message, click Yes.
6.	The Command window will open up and start dumping a bunch of information.
7.	If you do not see the Command window, select Command from the View menu.
8.	At the bottom of the Command window is the area where you enter commands. Figure 6-2 shows the Command window entry area. Figure 6-2. The Command window command entry location
9.	Type the command lm and press Enter (this will list the loaded DLLs in the process).
10.	Look for ntdll, and if the string to the right is "(pdb symbols)," your environment variable is set correctly. A good output looks like (the line is wrapped to fit in the book): 7c900000 7c9b0000 ntdll (pdb symbols) \\Symbols\OSSymbols\ntdll.pdb\ 36515FB5D04345E491F672FA2E2878C02\ntdll.pdb
11.	If the string next to ntdll says "(export symbols)," your _NT_SYMBOL_PATH is incorrect. An incorrect setup will have a line similar to the following (wrapped): 7c900000 7c9b0000 ntdll (export symbols) C:\WINDOWS\system32\ntdll.dll
12.	If you have an error, double-check your environment variable carefully and make sure you didn't swap the R and V characters in the SRV string. If you change the environment variable, you'll have to restart WinDBG and proceed through these steps again. For more information about fixing symbol problems, see the section "Ensuring That Correct Symbols Are Loaded" later in this chapter.
13.	If you changed your _NT_SYMBOL_PATH environment variable, remember to update the _NT_EXECUTABLE_PATH environment variable to have the same string.

WinDBG Options and Windows

WinDBG works on a workspace metaphor where settings are stored based on the main EXE binary name. Every time you change anything on a workspace, which includes set breakpoints; symbol, source, and binary paths; and window layout, WinDBG prompts you to save the workspace whenever the workspace is about to close. It's probably in your best interest to always save the workspace. You can delete unused workspaces or clear specific items saved with a workspace by selecting any of the workspace management items from the File menu.

When you don't have any binary or minidump open, any settings you make are to the base workspace, which all subsequent workspaces inherit. Before you start using WinDBG, you'll want to set a few options in the base workspace to make your life a little easier.

Colors Are Cool

When doing the test to check your symbol setup, you probably noticed that WinDBG is verbose to the extreme. Everything goes to the Command window, and you can easily lose track of anything important. Simply loading a large process can result in over 100 lines of output before you execute any WinDBG commands. Fortunately, WinDBG now allows you to color various reasons for the output so that you can separate the wheat from the chaff. The color selections appear at the bottom of the Options dialog box, which you can access by selecting Options from the View menu.

The bad news is that the meaning of all the various color items is not quite documented. Some of the items that you can color, such as Enabled Breakpoint Background, are self-explanatory, but others, such as Error Level Command Window Text, only appear to be self-explanatoryI never have seen my chosen color. In reality, the most important highlighting you'll want is on any Trace.Write* method calls that your programs make. (Remember, the DefaultTraceListener calls the Windows native OutputDebugString API function internally.) You can get these important values displayed in a different color by setting Debuggee Level Command Window Text to a different color. I personally always choose green because that indicates goodness to me.

Another color item you'll want to change is the User-selected command window line text. Often I am looking through a very long set of output in the Command window, and I want to mark key data in a different color. That way, when I am scrolling through the Command window, I can find the key areas based on color. Personally, I use red.

Although setting the color is easy, the steps required to change the color are not apparent. In the Command window, you'll highlight the text you want to change. In nearly every other application in the world, you'd right-click and select a menu option, such as Set Selection To User Colors. If you right-click selected text in the WinDBG Command window, that will put the selected text on the clipboard.

However, if you right-click the title bar of the Command window, you'll see the shortcut menu you would have expected inside the window. On that menu, you'll see the real Set Selection To User Colors command that will change the color. There are other items on the title bar shortcut menu, but the most important other than Set Selection To User Colors are Go To Mark (known in the rest of the world as a bookmark) and Help.

As is typical of WinDBG, although you can now set the selected text to your user colors, there's no way to quickly set it back to the default colors. To reset the colors, select Choose Color Text And Recolor Selection, and then manually set the color in the Color dialog box that pops up. I warned you that WinDBG was going to be interesting to learn!

Working with Windows

Before I dive into various WinDBG commands and SOS, I need to spend a few moments preparing you for what you're going to see with WinDBG. The WinDBG UI certainly reflects the fact that true debugger developers worked on it. It's functional, but it may not function as any Windows application you've ever seen.

The various WinDBG windows, which are all accessible on the View menu, are somewhat like tool windows in Visual Studio. When first opened, they default to being undocked from the main frame window and can be moved anywhere on your screen or multi-monitor system. If you double click a window's title bar, it will dock in the main frame. Dragging a window to the main frame will allow you dock the window in various positions in the main frame. Unlike Visual Studio's, the WinDBG windows will dock where they want to dock and not exactly where you want them to be.

The default docking options are to split up the main frame with multiple windows. If you're patient when dragging a window around the main frame, you'll eventually see the black frame fill up the entire main frame instead of just a portion. A window docked in the full frame will show up as a tab at the bottom on the main frame so you can quickly switch between each window. Figure 6-3 shows WinDBG set up with all the open windows docked in the same frame. It's a little odd to get the windows set with the stack at the bottom, which is how I like to set up WinDBG. The trick is to drag the window you want to dock completely away from WinDBG and drag it back to the middle of the WinDBG window. When the drag highlight frame surrounds the complete dock area, release the mouse button.

Figure 6-3. WinDBG windows showing tabbed docking

I go through the pain of getting my windows tab docked for two reasons: The first is that with the windows tiled inside the main window, it takes up a huge amount of screen real estate and you find yourself scrolling all the time. The second reason is due to a fundamental problem with WinDBG's floating windows: they have a serious z-order problem. If you have a floating window above the main frame, the floating window is always on top, so it blocks access to WinDBG menus and other windows. Some day, the WinDBG team will fix this glaring flaw.

If you have multiple monitors with plenty of screen real estate, you may just want to position the various WinDBG windows on your second monitor and avoid docking in the main frame all together. If you are really into window docking, you can open up multiple dock locations to anchor WinDBG windows where you want on your screen by selecting Open Dock from the Window menu. When you save a WinDBG workspace, you can save all the docks in addition to the window layouts.

The last two things I want to mention about WinDBG windows are tips to make your life easier. The first is that WinDBG always wants to show you the Disassembly window every time you do anything in WinDBG. To prevent the Disassembly window from appearing, on the Window menu, clear the check mark next to Automatically Open Disassembly.

The other window tip is if you look closely at Figure 6-3, you'll see that one of the tabs is titled Scratch Pad. As the name implies, WinDBG has a nice little scratch pad where you can copy and paste interesting data any time you want or if you're too lazy to Alt+Tab to Notepad. In order to save the data in the Scratch Pad window, you'll need to associate the Scratch Pad with a file by right-clicking the window's title bar and selecting Associate With File on the shortcut menu.

Dealing with Debuggees

Now that you have a good idea of some of the WinDBG basics, its time to turn to getting debuggees loaded into WinDBG so you can actually do some debugging. For your Windows and console applications, you can start them easily either by passing the file name of the executable on the WinDBG command line or selecting Open Executable on the File menu. If you're starting debugging on your application by passing it on the WinDBG command line, WinDBG treats any command-line arguments passed after the name of your process as such. The Open Executable dialog box, shown in Figure 6-4, shows the Arguments and Start Directory edit controls where you can specify either.

Figure 6-4. The WinDBG Open Executable dialog box

One major recent improvement to WinDBG is that it remembers the command line and starting directory you typed into the Open Executable dialog box when you restart debugging from the same instance of WinDBG. However, if you stop debugging in WinDBG and want to restart your program with a different set of command-line arguments, you'll search the Help in vain for any way to change the command line. You'll need either to end the current instance and restart a new instance of WinDBG or select Stop Debugging from the Debug menu to close the current instance. In both cases, you'll go back to the Open Executable dialog box to set command-line arguments and working directory. No one ever said WinDBG was the most user-friendly application in the world.

If your process is already running, you can also attach to that process by starting WinDBG and selecting Attach To A Process from the File menu. That menu will bring up the Attach To Process dialog box, in which you simply select the process you're interested in debugging and press OK. An extra treat in the Attach To Process dialog shows you the Terminal Services session for the process, the user account, and command line. If the process is running as a Windows Service, you'll see the name of the services running in the process.

No matter if you start or attach to a process, WinDBG will stop immediately in the debugger. This is different from Visual Studio, which has the started or attached process continue executing until it hits a breakpoint, has an unhandled exception, or you manually break into the process. If you don't want to stop when you start or attach to processes, you can change the Create Process event in selecting Event Filters from the Debug menu. This stop on start or attach is called the loader or initial breakpoint because in native debugging, the first instruction executed is a CPU-specific breakpoint so that debuggers can initialize their states and load items such as symbol tables. Alternatively, if you start WinDBG with the - g command-line option, it will skip the loader breakpoint so you immediately start your application.

There's one other debuggee type to discuss: minidumps. To open a minidump from the command line, pass -z followed by the name of the minidump file. Alternatively, select Open Crash Dump on the File menu. Once you have the minidump loaded, you can issue commands just as if you were sitting at the actual process.

When you're finished debugging, you can click the WinDBG main window Close button (X) at any time. If you're at the command prompt, the Q (Quit) command does the same thing. Both ways to quit will terminate the debuggee. If you want to leave the debuggee running but stop debugging, use the QD (Quit and Detach) command. The WinDBG default is to ask you to save the workspace every time you quit WinDBG. There's an option in the prompt dialog box that pops up when you stop debugging that asks if you want to turn off the dialog box permanently so that WinDBG doesn't ask you to save a workspace again. Personally, I want WinDBG to prompt me because I'm sure I'll forget to manually save the workspace the one time I really need it.