Tips and Tricks

[Previous] [Next]

In Chapter 5, I presented some tips and tricks that can make your life in the debugger easier. In this chapter, I'll continue to provide you with suggestions, but here I'll concentrate on tips that will help you with debugging at the assembly-language level.

Endians

The "Endianness" of a CPU refers to which end of a byte is stored first. Intel CPUs are "Little Endian," which means that the little end of a multibyte value is stored first. For example, the value 0x1234 is stored in memory as 0x34 0x12. It's important that you keep the Little Endian storage in mind when you're looking at memory in the debugger. You'll need to convert it in your head so that you're interpreting the correct values. If you use the Memory window to look at one of your link list nodes and the next pointer value is 0x12345678, the value will be displayed in byte format as 0x78 0x56 0x34 0x12.

If you're curious, the term "Endian" comes from Jonathan Swift's Gulliver's Travels, and the computer meaning came from a 1980 Request for Comments (RFC) concerning byte ordering by Danny Cohen. Danny's paper is at http://www.op.net/docs/RFCs/ien-137 for those who want to know the entire story.

Garbage Code

As a crash dumps you into the Disassembly window, you have to determine whether or not you're looking at real code, which is sometimes a difficult task. Here are some tips that will help you figure out whether you're looking at something other than executable code.

  • I've found that turning on the Code Bytes from the Disassembly window right-click menu to see the opcodes for the instructions is useful. As you'll see in the following tips, knowing what opcode patterns to look for can help you decide whether you're looking at legitimate code.
  • If you're looking at a series of identical ADD BYTE PTR [EAX] , AL instructions, you're not looking at valid assembly-language code. You're looking at a series of zeros.
  • If you see symbols but the offsets added to the symbols are very large numbers, generally over 0x1000, you're probably outside a code section. However, very large numbers can also mean that you're debugging a module that doesn't have private symbols available.
  • If you're looking at a bunch of instructions that I didn't cover in this chapter, you're probably looking at data.
  • If the Visual C++ disassembler can't disassemble an instruction, it displays "???" as the opcode.

Registers and the Watch Window

The Visual C++ debugger Watch window knows how to decode all the registers to values. Therefore, you can put a register in the Watch window and cast it to the type you want to observe. For example, if you're looking at a string manipulation instruction, you can enter (char*)@EDI in the Watch window to view the data in a format that's easier to read.

Learn from ASM Files

If you'd like to see more mixed assembly language and source code, you can have Visual C++ generate the assembly listings for your source files. If you type the /FAs option into the Project Settings dialog box, C++ tab, Project Options edit control, the compiler will generate an ASM file for each of your source code files. You might not want to generate the ASM files for every build, but they can be instructive, letting you see what the compiler is generating. The ASM files don't require you to fire up your application every time you're curious about assembly language.

The files generated are nearly ready to compile with the Microsoft Macro Assembler (MASM), so they can be a challenge to read. Much of the files consist of MASM directives, but the main parts of the files show your C code with the assembly-language code below each construct. After reading this chapter, you shouldn't have any trouble following the flow of ASM files.



Debugging Applications
Debugging Applications for MicrosoftВ® .NET and Microsoft WindowsВ® (Pro-Developer)
ISBN: 0735615365
EAN: 2147483647
Year: 2000
Pages: 122
Authors: John Robbins

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net