The version of Application Verifier that is integrated into Visual Studio 2005 is context sensitive. In other words, the option will only appear in the Debug menu if you create or load a C or C++ project. To start Application Verifier, select Debug Start with Application Verifier (also accessible by clicking Shift+Alt+F5).
To access the configuration panel for Application Verifier, click on your project in the Solution Explorer, and then select Properties Configuration Properties Application Verifier. The Application Verifier Properties Page window will appear, as shown in Figure 10-1.
The project property page has two setting categories: Heap Layer Advanced Settings and Verification Layers Settings. You can configure two settings within the Heap Layer Advanced Settings:
Conserve Memory: This option accepts two possible values: Yes or No. Guard pages are memory bookends that are placed at the beginning or the end of your application's memory allocation in order to detect any access violations. They are great for uncovering issues such as accesses after the end of the allocated memory block. If you select the No option, guard pages will be inserted in memory and a full page heap will be implemented. As a result, you may find that your overall system performance will temporarily degrade (the overhead is an acceptable trade-off for properly debugging your application). If you don't want memory constraints on your system, select the Yes option to implement a light page heap and conserve memory. However, keep in mind that a light page heap will not catch all of the problems, and finding the root cause of the problem may be difficult
Protection Location: The two values permitted here include "Protection at the beginning of the allocation" and "Protection at the end of the allocation." In a nutshell, these settings test for buffer underruns (at the beginning) and buffer overruns (at the end). Buffer overruns are a very common problem in unmanaged applications (buffer underruns are a less common problem). Setting the protection at the beginning of an allocation can help you detect issues such as memory accesses before the start of the allocated application memory block. You might be wondering at this point why protection location doesn't place a guard page at both the beginning and end of memory. When full page heap is used (Conserve Memory is set to No), the allocations are aligned with the page boundary. For example, if the application allocates 8 bytes, these 8 bytes are aligned with the end of the page if "Protection at the end of the allocation" is selected, and they are aligned with the beginning of the page if "Protection at the beginning of the allocation" is selected. If there is a buffer overrun or underrun, it may still be in the same page if they are only by a few bytes, which they usually are. So, putting a guard page at the beginning and end of the allocation at the same time is not going to help.
If you are interested in how the heap is structured, Team System's documentation has a great overview of both the light (or normal) heap and the full heap. Search for a document entitled "The Structure of a Page Heap Block" for more details.
You can set up several options for each verification layer on the configuration page:
Disable: The disable option will turn off the selected verification layer. Note that if you select Disable, you will not get the full range of run-time tests on your application. Disable a verification layer only when you want to isolate a problem or when it's absolutely required.
Enable–On Error Break and Log: By default, this option will be selected for any verification layer you choose to enable. If this option is selected, Application Verifier will show a Verifier stop message and break in to the debugger when an error is detected. It also logs a new item in the Task List (you'll learn more about this feature at the end of the chapter). The Tools window will also provide information to help you evaluate bugs.
Enable–On Error Break: This option will also enable a verification layer and break when an error is encountered. Unlike the previous option, no tasks will appear in the Task List as a result of an error.
Enable–On Error Log and Continue: This option enables a verification layer, creates an entry in the Task List when an error is encountered, and then the test run will resume (you will not see a Verifier stop message).
Application Verifier uses three verification layers to detect both memory-related and non-memory- related issues. You can enable or disable them at will using the Verification Layer Settings. The three layers include the following:
Heap Verification: The heap verification component of Application Verifier looks for corruption issues related to the portion of memory allocated by the running application.
Handle Verification: The Handle check ensures that your application doesn't use an invalid handle. This test will also verify that Thread Local Storage (TLS) indexes passed to TLS functions are valid, APIs are not called with handles that have been closed, and handles are valid when passed to APIs that take a handle. If the Handle Verifier encounters a handle with an INVALID_HANDLE_VALUE value or a NULL value, Application Verifier will break and provide you with information to investigate the problem.
Lock Verification: The Lock Verifier checks for the correct use of the critical sections. Please refer to the Troubleshooting Locks section to learn more about critical sections.
You'll learn more about these common errors in the rest of this chapter.