Windows 2000 Dr. Watson Log

[Previous] [Next]

Here's the first section of the Dr. Watson log from Windows 2000:

 Microsoft (R) Windows 2000 (TM) Version 5.00 DrWtsn32 Copyright (C) 1985-1999 Microsoft Corp. All rights reserved. Application exception occurred:     App: (pid=252)     When: 9/4/1999 @ 16:43:56.173     Exception number: c0000005 (access violation) 

The header information tells you what caused the crash—in this case, an application exception. The exception numbers for some crashes might not get translated into a human-readable description, such as "access violation" for exception number 0xC0000005. You can see all the possible exception number values by searching for STATUS_ in WINNT.H. The crash values are documented as EXCEPTION_ values returned by the GetExceptionCode function, but the real values are in all the STATUS_ #defines. Once you translate back into the EXCEPTION_ value, you can look up the description for the crash in the GetExceptionCode documentation.

The System Information section should be self-explanatory:

*----> System Information <----*         Computer Name: PLATO         User Name: John         Number of Processors: 1         Processor Type: x86 Family 6 Model 6 Stepping 10         Windows 2000 Version: 5.0         Current Build: 2128         Service Pack: None         Current Type: Uniprocessor Free         Registered Organization: Enter your company name here         Registered Owner: John Robbins

The Task List section looks like this:

*----> Task List <----*    0 Idle.exe    8 System.exe  132 smss.exe  160 csrss.exe  156 winlogon.exe  208 services.exe  220 lsass.exe  364 svchost.exe  424 svchost.exe  472 spoolsv.exe  504 MWMDMSVC.exe  528 MWSSW32.exe  576 regsvc.exe  592 MSTask.exe  836 Explorer.exe  904 tp4mon.exe  912 tphkmgr.exe  920 4nt.exe  940 taskmgr.exe  956 tponscr.exe  268 msdev.exe  252 WDBG.exe  828 NOTEPAD.exe  416 drwtsn32.exe    0 _Total.exe

The Task List section shows the processes that were running at the time of the crash. Unfortunately, Windows 2000 doesn't show the version information, so you'll have to ask the user for the file versions of all the processes in this section. The numbers down the left-hand side are the decimal process IDs (PIDs) at the time of the crash. The numbers are worthless after the fact.

This list contains all the modules loaded into the address space at the time of the crash:

(00400000 - 0042D000) D:\Dev\Book\CD\SourceCode\Output\WDBG.pdb (77F80000 - 77FF9000) E:\WINNT\symbols\dll\ntdll.dbg (60000000 - 6001A000) D:\Dev\Book\CD\SourceCode\Output\BugslayerUtil.pdb (77E80000 - 77F35000) E:\WINNT\symbols\dll\kernel32.dbg (77E10000 - 77E74000) E:\WINNT\symbols\dll\user32.dbg (77F40000 - 77F7C000) E:\WINNT\symbols\dll\gdi32.dbg (72950000 - 72967000) E:\WINNT\symbols\dll\dbghelp.dbg (78000000 - 78046000) (77DB0000 - 77E07000) E:\WINNT\symbols\dll\advapi32.dbg (77D30000 - 77DA2000) E:\WINNT\symbols\dll\rpcrt4.dbg (10200000 - 10264000) (63100000 - 63108000) D:\Dev\Book\CD\SourceCode\Output\LocalAssist.pdb (62000000 - 6202B000) D:\Dev\Book\CD\SourceCode\Output\i386CPUHelp.pdb (63000000 - 63010000) D:\Dev\Book\CD\SourceCode\Output\LocalDebug.pdb (5F400000 - 5F4E5000) (77B30000 - 77BBA000) E:\WINNT\symbols\dll\comctl32.dbg (775A0000 - 777DE000) E:\WINNT\symbols\dll\shell32.dbg (77C50000 - 77C9A000) E:\WINNT\symbols\dll\shlwapi.dbg (76B20000 - 76B5E000) E:\WINNT\symbols\dll\comdlg32.dbg (77A30000 - 77B24000) E:\WINNT\symbols\dll\ole32.dbg (77990000 - 77A24000) E:\WINNT\symbols\dll\oleaut32.dbg (77CA0000 - 77D25000) (77850000 - 7788B000) (770B0000 - 770D3000) (6B6E0000 - 6B6FC000) E:\WINNT\symbols\dll\msdbi.dbg (68ED0000 - 68EDB000) E:\WINNT\symbols\dll\psapi.dbg

The numbers are in the format (load address - maximum address) for each module. This section would be more useful if Dr. Watson showed the actual names of the modules loaded into the address space. Unfortunately, as you can see by the names to the right of the numbers, the only information shown is the location where Dr. Watson loaded the debugging symbols for that module. The blank entries mean that no debugging information was loaded for the module.

To figure out what modules were loaded, you have to guess. As I've mentioned several times in this book, it's vital for you to know where your dynamic-link libraries (DLLs) load into the process address space. You can probably recognize your DLLs just from the load addresses. To find the information for the other DLLs on the user's machine, you could write a small utility that would run through the DLLs on the user's system and report their names, load addresses, and sizes.

The following output is the beginning of the three-part thread state. (You'll notice that I removed the code bytes from beside the disassembly addresses and wrapped the register display so that it would fit on the page.)

State Dump for Thread Id 0x2fc eax=00000000 ebx=7ffdf000 ecx=008c67c0 edx=0000033c esi=00134c78 edi=0012fd74 eip=0040bd2d esp=0012fb98 ebp=0012fbc4 iopl=0 nv up ei pl nz na pe nc cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202 function: CWDBGProjDoc::HandleBreakpoint         0040bd11 push    esi         0040bd12 push    edi         0040bd13 mov     eax,0xcccccccc         0040bd18 mov     [ebp+0xe0],eax          ss:00b4d19a=????????         0040bd1b mov     [ebp+0xe4],eax          ss:00b4d19a=????????         0040bd1e mov     [ebp+0xe8],eax          ss:00b4d19a=????????         0040bd21 mov     [ebp+0xec],eax          ss:00b4d19a=????????         0040bd24 mov     [ebp+0xf0],eax          ss:00b4d19a=????????         0040bd27 mov     [ebp+0xe0],ecx          ss:00b4d19a=????????         0040bd2a mov     eax,[ebp+0xc]           ss:00b4d19a=???????? FAULT ->0040bd2d mov     ecx,[eax+0x4]           ds:00a1d5d6=????????         0040bd30 cmp dword ptr [ecx],0x80000003  ds:008c67c0=0041b714         0040bd36 jz CArray<COneShotBP,COneShotBP>::SetSize+0x25d (0041485d)         0040bd38 mov     esi,esp         0040bd3a push    0x382         0040bd3f push    0x420030         0040bd44 push    0x420064         0040bd49 push    0x0         0040bd4b call    dword ptr [_imp__DiagAssertA (00423ad4)]         0040bd51 cmp     esi,esp         0040bd53 call    _chkesp (00416b5a)         0040bd58 test    eax,eax

Dr. Watson displays the state information for each thread running in a process at the time it crashed. The thread states contain all the information you need to determine how and why the system crashed.

The register portion shows what all the registers contained at the time of the crash. The important register to look at is EIP, the instruction pointer. The example I show has symbols, so you can see which function this thread was executing when the crash occurred—most Dr. Watson logs won't have symbol information. Of course, it's not a problem if Dr. Watson doesn't give you the function name. Using CrashFinder from Chapter 8, simply load up the CrashFinder project for your application, press Ctrl+F, and enter the EIP for the thread in the Find dialog box.

This thread happened to be the thread that crashed. The only indicator is the FAULT-> pointer in the middle of the disassembly. I've seen a Dr. Watson log or two that didn't display the FAULT-> pointer. If you don't see this pointer in a log, run through each thread state and enter each EIP address into CrashFinder to figure out where the thread was sitting at the time of the crash.

The disassembly should look familiar to you if you remember what you read in Chapter 6. The only new elements are the values after the instructions. The Dr. Watson disassembler attempts to look up the effective address of the memory reference for the instruction so that you can see what value the instruction was manipulating. The addresses that start with ss indicate that the memory was a stack segment reference; ds indicates a data segment reference.

The only effective address in the disassembly that's guaranteed to be correct is the one at the instruction pointer. The others might be incorrect because the value the instruction refers to could have been changed. For example, let's say that the first instruction disassembled in the thread state had a memory reference at EBX. If the crash instruction occurred 10 instructions later, one of the intervening instructions could easily have changed EBX. When Dr. Watson does its disassembly, however, it uses the value currently in EBX—that is, the one at the time of the crash—to do the effective address translation. For this reason, the effective address shown in the disassembly could be incorrect. Carefully check whether any instructions could change the register values before you believe what you see in the effective address display.

Using your newfound assembly-language skills, you should be able to figure out that this thread crashed because the instruction at 0x0040BD2D, MOV ECX , [EAX+0x4] was trying to access a NULL pointer in EAX. If you look at the instruction on the line just above this one, MOV EAX , [EBP+0xC], the light bulb should go on in your head that positive offsets from EBP are probably parameter references. An offset of 0xC from EBP means that the instruction referenced the second parameter to the function. Your first debugging hypothesis for this crash should be that the function failed because an invalid parameter was passed in the second parameter. (I hope this paragraph has convinced you how important it is to know enough assembly language to read a Dr. Watson log!)

Here's the second part of the thread state: the Stack Back Trace. (Notice that I wrapped the function names so that everything would fit on the page.)

*----> Stack Back Trace <----* FramePtr ReturnAd Param#1 Param#2 Param#3 Param#4 Function Name 0012FBC4 0040BCB5 0000033C 00000000 80000003 008C67C0                                           !CWDBGProjDoc::HandleBreakpoint 0012FBE0 00405A9C 0000033C 010DFC0C 008C68B0 0012FCF4                                      !CWDBGProjDoc::HandleExceptionEvent 0012FBF4 5F42F3AC 0000033C 010DFC0C 0012FD74 00134C78                                      !CDocNotifyWnd::HandleExceptionEvent 0012FCF4 5F42ECE8 00000502 0000033C 010DFC0C 0012FD10 !Ordinal4118 0012FD14 5F42C889 00000502 0000033C 010DFC0C 77F86618 !Ordinal5076 0012FD88 5F42CD25 008C68B0 000602A4 00000502 0000033C !Ordinal1045 0012FDB4 5F4905FD 000602A4 00000502 0000033C 010DFC0C !Ordinal1192 0012FDE4 77E135F8 000602A4 00000502 0000033C 010DFC0C !Ordinal1193 0012FE04 77E15FE8 5F4905B3 000602A4 00000502 0000033C   user32!UserCallWinProc 0012FE20 77E1600E 004A9B70 00000502 0000033C 010DFC0C   user32!DispatchClientMessage (FPO: Non-FPO [5,1,0]) 0012FE48 77F9D8B7 0012FE58 00000018 004A9B70 00000502   user32!__fnDWORD (FPO: Non-FPO [1,4,0]) 0012FE68 77E15FB5 77E17BD9 00422250 00000000 00000000   ntdll!KiUserCallbackDispatcher (FPO: [0,0,0]) 0012FE90 5F4396F8 00422250 00000000 00000000 00000000   user32!DispatchClientMessage (FPO: Non-FPO [5,1,0]) 0012FEB8 5F438E1D 77F86618 77F81A9B 7FFDF000 00000001 !Ordinal4239 0012FEDC 5F439AD4 00422218 0012FF08 5F43366E 77F86618 !Ordinal4409 0012FEE8 5F43366E 77F86618 77F81A9B 7FFDF000 00422218 !Ordinal4408 0012FF08 00417028 00400000 00000000 00133A73 00000001 !Ordinal1190 0012FF20 00416E53 00400000 00000000 00133A73 00000001 !WinMain 0012FFC0 77E9BC52 77F86618 77F81A9B 7FFDF000 C0000005 !WinMainCRTStartup 0012FFF0 00000000 00416CA0 00000000 000000C8 00000100   kernel32!BaseProcessStart (FPO: Non-FPO [1,8,3])

Although my example Dr. Watson log has symbols, your user's log probably won't. The ReturnAd column lists the return addresses of functions on the call stack. If your user's log doesn't have symbols, all you need to do is load each address in the ReturnAd column into CrashFinder to find out the sequence of function calls leading up to the crash.

The Param# columns show a function's first four possible parameters on the stack. With highly optimized release builds and no symbols, the values are probably incorrect. However, you can still use them as a starting point for hand-walking your code.

The function names are shown in <module>!<function> format. Those functions shown as Ordinal# are ordinal exports. If you don't have the source code for the DLL that does the exports by ordinal, you're mostly out of luck. However, because you do have the source code for the Microsoft Foundation Class (MFC) library, you can look up MFC ordinal values. In the WDBG program, I know that MFC42D.DLL is loaded at 0x5F400000, so I can look up those ordinals because all MFC functions are exported by ordinal value through a linker definition (DEF) file.

The one prerequisite for converting MFC ordinal values to functions is that you must be certain of the version of the MFC DLL on the machine that crashed. On my machine, named \\PLATO, I had MFC42D.DLL from Visual C++ 6 Service Pack 3. If you're unsure of the version of MFC on the user's machine, you'll either have to ask for it or beg for it—I'll let you choose the method you're most comfortable with.

Follow these simple steps to turn ordinal values into functions:

1. Open the <Visual C++ directory>\MFC\SRC\Intel directory.
2. Select the appropriate DEF file for the MFC file you want to look up. For example, MFC42D.DLL's DEF file is MFC42D.DEF.
3. Search for the ordinal number. To find Ordinal4118 from the preceding stack, I'd search the MFC42D.DEF file for 4118. The line with 4118 is "?OnWndMsg@CWnd@@MAEHIIJPAJ@Z @ 4118 NONAME."
4. The name to the left of "@ 4118 NONAME" is the decorated function name exported at that ordinal value. To undecorate the name, use the UNDNAME.EXE program that comes with the Platform SDK. For 4118, the function is CWnd::OnWndMsg.

The third part and final part of the thread state is the Raw Stack Dump:

*----> Raw Stack Dump <----* 0012fb98 74 fd 12 00 78 4c 13 00-00 f0 fd 7f c0 67 8c 00 t...xL.......g.. 0012fba8 cc cc cc cc cc cc cc cc-cc cc cc cc cc cc cc cc ................ 0012fbb8 e8 fc 12 00 2e 8a 41 00-ff ff ff ff e0 fb 12 00 ......A......... 0012fbc8 b5 bc 40 00 3c 03 00 00-00 00 00 00 03 00 00 80 ..@.<........... 0012fbd8 c0 67 8c 00 cc cc cc cc-f4 fb 12 00 9c 5a 40 00 .g...........Z@. 0012fbe8 3c 03 00 00 0c fc 0d 01-b0 68 8c 00 f4 fc 12 00 <........h...... 0012fbf8 ac f3 42 5f 3c 03 00 00-0c fc 0d 01 74 fd 12 00 ..B_<.......t... 0012fc08 78 4c 13 00 00 f0 fd 7f-09 00 00 00 b0 68 8c 00 xL...........h.. 0012fc18 04 00 00 00 00 00 00 00-00 00 8c 00 50 00 8c 00 ............P... 0012fc28 50 00 8c 00 01 00 00 00-ae 05 12 00 70 2d 00 00 P...........p-.. 0012fc38 01 00 00 00 fa 00 00 00-8c fc 12 00 8b 48 fb 77 .............H.w 0012fc48 00 00 8c 00 00 00 9f 01-01 00 00 00 80 fc 12 00 ................ 0012fc58 7c fc 12 00 84 fc 12 00-00 00 00 00 00 00 00 00 |............... 0012fc68 00 00 8c 00 00 00 8c 00-90 fc 12 00 d5 19 49 5f ..............I_ 0012fc78 04 00 00 00 18 66 f8 77-9b 1a f8 77 a4 fc 12 00 .....f.w...w.... 0012fc88 d5 19 49 5f 04 00 00 00-18 66 f8 77 9b 1a f8 77 ..I_.....f.w...w 0012fc98 00 f0 fd 7f 48 e1 4c 5f-48 4d 13 00 cc fc 12 00 ....H.L_HM...... 0012fca8 45 16 49 5f cc fc 12 00-d5 19 49 5f d4 fc 12 00 E.I_......I_.... 0012fcb8 d8 fc 12 00 d5 19 49 5f-04 00 00 00 18 66 f8 77 ......I_.....f.w 0012fcc8 50 22 42 00 00 00 00 00-0a 00 00 00 00 a5 41 00 P"B...........A.

I rarely look at this information. If I'm really stuck on a crash, however, I might start looking at the information to see whether I can guess at local variable values. The two return addresses I can correlate with the preceding stack walk are shown in boldface.