Inside the UnhandledExceptionFilter Function

[Previous] [Next]

When I first started working with exceptions, I believed that a lot of information could be gained from understanding exactly what the system's UnhandledExceptionFilter function did. So, I researched this function in great detail. The following steps describe exactly what UnhandledExceptionFilter does internally:

1. If an access violation occurred and it was due to an attempted write (versus a read), the system checks to see if you're attempting to modify a resource in an .exe or DLL module. By default, resources are (and should be) read-only; attempting to modify a resource raises an access violation. However, 16-bit Windows allows resources to be modified, and for backward compatibility this should work in 32-bit and 64-bit Windows as well. Thus, if you are attempting to modify a resource, UnhandledExceptionFilter calls VirtualProtect to change the protection on the resource's page to PAGE_READWRITE and returns EXCEPTION_CONTINUE_EXECUTION.
2. If you have called SetUnhandledExceptionFilter to specify your own filter, UnhandledExceptionFilter calls your filter function. If your filter function returns EXCEPTION_EXECUTE_HANDLER or EXCEPTION_CONTINUE_EXECUTION, the UnhandledExceptionFilter returns this value back to the system. If you have not set your own unhandled exception filter or if your unhandled exception filter returns EXCEPTION_CONTINUE_SEARCH, processing continues with step 3.

Windows 98

---

Windows 98 has the following bug: it only calls your own unhandled exception filter function if the process is not being debugged. This made it impossible for me to debug the Spreadsheet sample application presented later in this chapter.

3. If your process is under the care of a debugger, EXCEPTION_CONTINUE_SEARCH is returned. This should seem odd to you because the system is already executing the highest try or except frame for the thread; there is no other exception filter to continue searching for higher up. When the system sees that the highest filter returns EXCEPTION_CONTINUE_SEARCH, the system knows to contact the debugger and tell the debugger that the debuggee has just had an unhandled exception. In response, the debugger displays a message box and allows you to debug the process. (By the way, the IsDebuggerPresent function is used to determine if a process is being debugged.)
4. If a thread in the process had called SetErrorMode passing the SEM_NOGPFAULTERRORBOX flag, UnhandledExceptionFilter returns EXCEPTION_EXECUTE_HANDLER.
5. If the process is in a job (see Chapter 5) and the job's limit information has the JOB_OBJECT_LIMIT_DIE_ON_UNHANDLED_EXCEPTION flag turned on, UnhandledExceptionFilter returns EXCEPTION_EXECUTE_HANDLER.

Windows 98

---

Windows 98 doesn't support jobs so this step is skipped.

6. UnhandledExceptionFilter looks in the registry and grabs the value of the Auto value. If this value is 1, jump to step 7. If this value is 0, the message box is displayed to the user. The message box indicates which exception was raised. If the registry subkey also contains the Debugger value, the message box has both OK and Cancel buttons. If the registry subkey does not have the Debugger value, the message box contains only an OK button. If the user clicks on the OK button, UnhandledExceptionFilter returns EXCEPTION_EXECUTE_HANDLER. If the Cancel button is available and the user clicks on it, processing continues with step 7.

Windows 98

---

In Windows 98, these values are not stored in the registry—they are stored in the Win.ini file.

7. UnhandledExceptionFilter is now going to spawn the debugger. It first calls CreateEvent to create a nonsignaled, manual-reset event. The handle of this event is inheritable. Next it grabs the Debugger value out of the registry and calls sprintf to paste in the process ID (obtained by calling the GetCurrentProcessId function) and the event handle. The STARTUPINFO's lpDesktop member is also set to "Winsta0\\Default" so that the debugger appears on the interactive desktop. CreateProcess is then called with its fInheritHandles parameter set to TRUE, which invokes the debugger process and allows it to inherit the event object's handle. UnhandledExceptionFilter now waits for the debugger to initialize by calling WaitForSingleObjectEx, passing the event's handle. Note that WaitForSingleObjectEx is used instead of WaitForSingleObject so that the thread can wait in an alertable state. This allows any queued asynchronous procedure calls (APCs) for the thread to be processed.
8. When the debugger is fully initialized, it sets the event handle, which causes the thread inside UnhandledExceptionFilter to wake up. Now that the process is under the care of a debugger, UnhandledExceptionFilter returns EXCEPTION_CONTINUE_SEARCH. Notice that this is exactly what happened in step 3.