Advanced Tips and Tricks

In this section, I want to discuss the features that the debugger has up its sleeve to make debugging easier. Half of the debugging process is knowing how to use your tools better, and Visual Studio is filled with features to help you solve your problems quickly and efficiently.

The Set Next Statement Command

One of the coolest hidden features in the Visual Studio debugger is the Set Next Statement command. It is accessible in both source windows and the Disassembly window on the shortcut menu or by pressing Ctrl+Shift+F10 (using the default keyboard layout), but only when you're debugging. What the Set Next Statement command lets you do is change the instruction pointer to a different place in the program. Changing what the program executes is a fantastic debugging technique when you're trying to track down a bug or when you're unit testing and want to test your error handlers.

A perfect example of when to use Set Next Statement is manually filling a data structure. You single-step over the method that does the data insertion, change any values passed to the function, and use Set Next Statement to force the execution of that call again. Thus, you fill the data structure by changing the execution code.

I guess I should mention that changing the instruction pointer can easily crash your program if you're not extremely careful. For .NET applications, it's safe enough to use Set Next Statement at the source code level for both debug and release builds. However, if you skip over a variable initialization to a spot where that variable is used, you'll cause a NullReferenceException in your code.

If I'm looking for a bug, and my hypothesis is that said bug might be in a certain code path, I set a breakpoint in the debugger before the offending method or methods. I check the data and the parameters going into the methods, and I single-step over the methods. If the problem isn't duplicated, I use the Set Next Statement command to set the execution point back to the breakpoint and change the data going into the methods. This tactic allows me to test several hypotheses in one debugging session, thus saving time in the end. As you can imagine, you can't use this technique in all cases because once you execute some code in your program, executing it again can destroy the state. Set Next Statement works best on code that doesn't change the state too much.

As I mentioned earlier, the Set Next Statement command comes in handy during unit testing. For example, Set Next Statement is useful when you want to test error handlers. Say that you have an if statement and you want to test what happens when the condition fails. All you need to do is let the condition execute and use Set Next Statement to move the execution point down to the failure case. In addition to Set Next Statement, the Run To Cursor menu option, also available on the shortcut menu in a source code window when debugging, allows you to set a one-shot breakpoint. I also use Run To Cursor quite a bit in testing, and it is especially useful to skip long loops without creating a breakpoint that might trigger afterwards and disturb your mind from the workflow you are interested in.

Filling data structures, especially lists and arrays, is another excellent use of Set Next Statement when you're testing or debugging. If you have some code that fills a data structure and adds the data structure to a list, you can use Set Next Statement to add some additional items to the list so that you can see how your code handles those cases. This use of Set Next Statement is especially handy when you need to set up difficult-to-duplicate data conditions when you're debugging.

You can get quite creative with Set Next Statement. In one debugging battle, we needed to look at a file the debuggee had opened with no sharing. I used Set Next Statement to move the instruction pointer to the Close method call, single stepped over the Close, and therefore was able to look at the file because the sharing lock was removed and could even modify the contents of the file. After I was finished looking at the data, I used Set Next Statement back to the line that opened the file and reset the state of the application.

Mixed-Mode Debugging

For those of you doing interop to native code, which includes most .NET developers, one of the major improvements with .NET 2.0 and Visual Studio 2005 is the massive performance boost for mixed-mode debugging, which allows you to debug both the managed and native parts at the same time. In prior versions of Visual Studio, the mixed mode debugging performance was so abysmal that you were far better off debugging both sides of the application by themselves. In some cases with Visual Studio 2005, I've turned on mixed-mode debugging and forgotten for weeks that it's on. My only complaint is that mixed-mode debugging works only on 32-bit operating systems. For 64-bit systems, you'll need to debug each side of the application separately.

Part of the confusion of mixed-mode debugging is getting it turned on because the option location is different depending on the type of project you're working on. For console and Windows Forms applications, you'll go into the project properties, select the Debug property page, and at the bottom of the page, select the Enable Unmanaged Code Debugging check box. Figure 5-14 shows the location.

Figure 5-14. Enabling unmanaged debugging in a Console or Windows Forms application

For ASP.NET applications, the option for turning on unmanaged debugging is hidden in the Start Options property page, which you can get to by right-clicking the Web site and selecting Properties. In the ensuing Web Site Property Pages dialog box, select Start Options, and at the bottom of the property page is the Debuggers section. Selecting the Native Code check box turns on unmanaged debugging. Figure 5-15 shows the location.

Figure 5-15. Enabling unmanaged debugging in an ASP.NET application

Debugging .NET assemblies that load into native processes can be a bit confusing. For example, if you're developing Microsoft Management Console (MMC) snap-ins in C#, the confusion can occur because although MMC.exe is a native application, it can host the .NET runtime if your assembly implements the snap-ins COM interface. If the hosting executable can be started by the debugger, like MMC.exe, you'll go into the Debug property page and set the name of the process to run in the Start external process edit control. The Visual Studio debugger will behave as expected, and you'll be debugging your C# code with the managed debugger. To turn on both managed and native debugging, you'll select the Enable Unmanaged Debugging check box as I described earlier.

If you've opened the native hosting EXE by going to File, Open, and Project/Solution, life gets a little more interesting if you want to do both native and managed debugging. When opening an EXE outside of a solution, the IDE defaults to a C++ project. That means when you press any of the debugging keys to start debugging, the debugger inspects the binary, and if it's a native binary, it does only native debugging. Conversely, if it's a managed binary, the debugger does only managed debugging. To ensure that you get the exact type of debugging you want, right-click the EXE in Solution Explorer and select Properties from the shortcut menu. In the Property Page dialog box, the only option is the Debugging property page. In the Debugger Type field, the default is Auto, which uses the EXE type to determine the type of debugging to do. To do both native and managed debugging, in the Debugger Type list, select Mixed as shown in Figure 5-16.

Figure 5-16. Enabling mixed-mode debugging when debugging a stand-alone EXE

The final way you can enable mixed-mode debugging is when you attach to an existing process. In the Attach To Processes dialog box, accessible by selecting Attach To Processes from the Debug menu, the debugger, as it does when opening up just an EXE, attempts to determine the type of debugging you'll want to do. In order to change the default, click Select, and in the Select Code Type dialog box, select the Debug These Code Types option button, and then select both Managed and Native check boxes. Keep in mind that no matter how you turn on mixed-mode debugging, it can be turned on only when you first start or attach to the process. Unfortunately, there's no way to toggle mixed-mode debugging.

Once you are doing mixed-mode debugging, a new feature sneaks into the Debug menu when you break into the debugger: the ability to write minidumps while debugging, which you do by selecting the Save Dump As menu item. For this reason alone, you may want to consider always running mixed-mode debugging as writing out a dump so you can look at your application state after you finish debugging.

After you've selected the new menu item, the Save Dump As dialog box appears, and, unfortunately, always defaults to saving your dump in your user_name\Application Data directory. Personally, I wish the dialog box would remember the last directory where you saved a minidump file. One key item in the Save Dump As dialog box is the Save As Type drop-down list. It offers two types of dump: Minidump and Minidump with Heap. The straight Minidump type is what I refer to as a basic minidump. It contains the list of loaded modules, their load addresses, and just enough information to walk the native stack. Those minidumps are nice and small, but they sadly don't have sufficient information in them to see the managed side of your application.

A Minidump with Heap contains everything in a basic minidump, but it also contains all the actual memory for the program. As you can guess, those minidumps don't have anything mini about them, because they are the complete address space for the process. However, the Minidump with Heap is what you'll need to look at your managed portions.

Opening minidumps is almost as easy as creating them. You'll start a new instance of Visual Studio, and on the File menu, select Open, and then Project/Solution. In the Open Project dialog box, navigate to the directory where your minidump file is located and double-click it. That will create a new solution with your minidump the only item in it. Once the minidump is open, you'll press debugging keys, such as F5, to start debugging. Visual Studio will open the minidump and allow you to look at the native side of the application at the moment you wrote the minidump. To see the managed side, you'll have to load the Son of Strike (SOS) extension into the Immediate window and use it from there. Chapter 6, "WinDBG, SOS, and ADPlus" goes into the hard-core usage of SOS.

Debugging Exceptions

When developing, it's always an excellent idea to know exactly what code is throwing and catching exceptions. Fortunately, Visual Studio makes debugging exceptions quite easy. On the Debug menu, selecting Exceptions brings up the Exceptions dialog box in which you can tell the debugger exactly what to do when your code throws any exception. Figure 5-17 shows the Exceptions dialog box with some of the Common Language Runtime (CLR) Exceptions expanded.

Figure 5-17. Exceptions dialog box

The debugger team has drastically simplified the Exceptions dialog box compared to the one in previous releases. If you want to stop whenever your application throws a System.ArgumentNullException, for example, you would expand the Common Language Runtime Exceptions node and the System node, and then put a checkmark in the Thrown column next to System.ArgumentNullException. When your application and any assembly it is using now throws this error, you'll stop in the debugger with the Exception pop-up message, before any catch block is searched.

Once you're at the code that threw the exception, if you press the Step Into key (F11 on the default keyboard layout), you'll single-step to the exact catch block that's handling that particular exception. Because numerous bugs I've looked at involved overzealously catching exceptions, I know that gaining insight into exactly what code is handling particular exceptions can avoid those problems completely. What I like to do many times when debugging is place a check mark in the Thrown statement next to the Common Language Runtime Exceptions node. That way I'll stop in the debugger whenever I've had any type of exception so I can assure that the handlers are not too generic and are only handling just the exceptions they are supposed to handle.

The one drawback to the approach I just outlined is that you will now be stopping on exceptions that occur anywhere in the address space, including from the CLR and FCL itself. That can get to be a little tedious if you are just trying to see the exceptions your code throws. The reason is that I strongly recommended in the beginning of the chapter to turn off the Just My Code feature. However, this is one time where Just My Code does help you out. If it's turned on, and you set the Exceptions dialog box to stop on all thrown exceptions, you'll stop only inside your code. Personally, I never turn on Just My Code, because I do want to see every possible exception thrown anywhere in my application, but when you're working on very large applications, turning it on trying to look at your exceptions can speed you up.

The last trick with .NET exceptions I want to mention is something that's incorrectly documented in the Visual Studio documentation. If you read the documentation page, "Continuing Execution After an Exception," it says for managed code, "the Exception Assistant unwinds the call stack to the point where the exception was thrown." As anyone who's seen the Exception Assistant knows, that's not the case. The Exception Assistant shows you the location after the throw occurred. Fortunately, there's a nifty little trick to get the state of the application right back to the instruction causing the throw.

After you dismiss the Exception Assistant, go to the Call Stack window in the debugger. Right-click on the first line in the call stack, and the shortcut menu will have an additional menu item on it, Unwind To This Frame. It is active only if you're stopped because of a throw. Selecting Unwind To This Frame will reset the stack to the instruction and state at the time of the throw. This little trick will allow you to change parameters or locals in the debugger to avoid the exception in the first place.

Debugging Multiple Threads and Processes

Life would be so much simpler if we allowed only one thread in a process, but that's not the reality of modern development. Visual Studio has a fine Threads window, which, like all debugger windows, is accessible by selecting Debug on the Windows menu. When stopped in the debugger, the Threads window shows you the Windows thread ID, the name of the thread, its current location, the current priority, and the suspend count. Note that through .NET 2.0, a native thread backs each managed thread. That may change in future versions of .NET.

The most important trick for debugging multiple threads is to set the Thread.Name property so you'll see that name in the Threads window Name column. That makes identifying the threads much easier. Keep in mind that you can set the Thread.Name property only once because it generates an InvalidOperationException when you try to change the name. You'll want to always set those threads you create yourself, but you'll want to be very careful with ThreadPool threads in checking first if this property is null before setting it. Also, if you're worried about exposing too much of your internal details by naming a thread, you can use conditional compilation so you name only the threads in debug builds.

Of course, if you forget to name your threads in your code, because you all now know the secrets of the Watch window masters, you can simply name the threads by using the Watch window. Type System.Threading.Thread.CurrentThread.Name into the Watch window, and if the name value is null or Nothing, you can set it to whatever you want. If the name is already set, you'll generate an InvalidOperationException trying to set the name because it can be set only once.

One nice trick you have in the Threads window is that you can suspend threads by right-clicking them and selecting Freeze from the shortcut menu. Those threads will be marked with double blue bars in the first column of the Threads window and will have their suspend count incremented by one. Figure 5-18 shows two threads manually frozen in the Threads window. One trick with freezing threads I haven't seen mentioned is that you can select multiple threads and freeze them all with a single right-click. Thawing threads is as simple as right-clicking the frozen threads and selecting Thaw from the shortcut menu.

Figure 5-18. Frozen threads in the Threads window

As you can imagine, wantonly freezing your threads can cause a few issues in your application if you're not careful. However, it's a nice trick if you want to follow a single connection in an ASP.NET application through to completion. If you're doing mixed-mode debugging, you'll also want to be extra careful about which threads you freeze. If you freeze the finalizer thread, you'll cause a major problem for your application. In Chapter 6, you'll learn how to identify the finalizer thread with SOS.

The new Processes window, which was sorely missing in previous versions of Visual Studio, makes debugging multiple processes much easier. As with any of the debugging windows, you can access the Processes window by selecting Windows on the Debug menu once a debugging session is started. By default, when one process you are debugging breaks, all processes being debugged will stop in the debugger as well. That way if you have multiple processes talking to one another, all of them will stop at the same time. However, if you go into the IDE Options dialog box, Debugging General node, the second item down is Break All Processes When One Process Breaks. If you clear that option, you'll break only on the individual process that you have chosen to break into or on a process encountering a breakpoint or exception.

The great news is that you can toggle this option on and off all you want when debugging. If the Process window shows a green go button and a blue pause button as the first two buttons, you'll know that the option to break independently is active. Although you probably won't enable this option very much, it's nice that the debugger team made it an option for those times when you are debugging multiple disparate programs.