Calling Conventions

[Previous] [Next]

Before I move into other instructions, let's take a brief detour into calling conventions. The few instructions presented in the last section will help you do some excellent debugging. However, I need to tie procedure calling and calling conventions together so that I can start showing you how to decipher the Disassembly window.

A calling convention specifies how parameters are passed to a function and how stack cleanup occurs when the function returns. The programmer who codes a function dictates the calling convention that everyone must follow when calling that function. The CPU doesn't dictate any specific calling conventions. If you understand the calling conventions, you'll find it much easier to look up parameters in the Memory window and to determine the flow of the assembly language in the Disassembly window.

There are five calling conventions in all, but only three are common: the standard call (__stdcall), the C declaration (__cdecl), and the this call. Although you can specify the standard call and the C declaration yourself, the this call is automatically applied when you're using C++ code and dictates how the this pointer is passed. The other two calling conventions are the fast call (__fastcall) and the provocatively named naked calling conventions. By default, Win32 operating systems don't use the fast-call calling convention in user-mode code because it isn't portable to other CPUs. The naked calling convention is used in virtual device driver (VxD) programming and when you want to control the prolog and epilog generation yourself, as you'll see in Chapters 12 and 14.

Table 6-3 lists all the calling conventions. Recall from Chapter 5 the description of the name decoration scheme for setting breakpoints on system functions. In Table 6-3, you'll see that the calling convention dictates the name decoration scheme.

If you've never been exposed to the different calling conventions, you might wonder why the different types exist. The differences between the C declaration and the standard call are subtle. In a standard call function, the callee cleans up the stack, so it has to know exactly how many parameters to expect. Therefore, a standard call function can't be a variable argument function such as printf. Because C declaration functions have the caller cleaning up the stack, variable argument functions are just fine. Standard call is the default for Win32 system functions as well as for Visual Basic functions.

Table 6-3 Calling Conventions

Calling Convention	Argument Passing	Stack Maintenance	Name Decoration	Notes
__cdecl	Right to left.	Caller removes arguments from the stack. This calling convention is the only one that allows variable argument lists.	Underscore prefixed to function names, as in _Foo	The default for C and C++ functions.
__stdcall	Right to left.	Callee removes its own arguments from the stack.	Underscore prefixed to function name, and @ appended followed by the number of decimal bytes in the argument list, as in _Foo@12.	Used by almost all system functions; the default for Visual Basic internal functions.
__fastcall	First two DWORD parameters are passed in ECX and EDX; the rest are passed right to left.	Caller removes arguments from the stack.	An @ is prefixed to the name, and @ is appended followed by the number of decimal bytes in the argument list, as in @Foo@12.	Applies only to Intel CPUs. This calling convention is the default calling convention for Borland Delphi compilers.
this	Right to left. The this parameter is passed in the ECX register.	Caller removes arguments from the stack.	None	Used automatically by C++ class methods unless you specify standard call. COM methods are declared as standard call.
naked	Right to left	Caller removes arguments from the stack.	None	Used by VxDs and when you need custom prolog and epilog.

Calling Conventions Example

To tie together the instructions I've shown so far and the calling conventions, Listing 6-1 shows an example of all the calling conventions from the Visual C++ debugger's Disassembly window. The sample source code, CALLING.CPP, is on the companion CD if you want to take a look at it.

The code in Listing 6-1 is a debug build to make it easier to follow; also, the code doesn't actually do anything. I call each calling convention function in turn. Pay special attention to how the parameters are pushed to each function and to how the stack is cleaned up. I inserted NOP instructions between each of the functions to make the listing easier to read.

Listing 6-1 Calling conventions example

6: // The strings passed to each function 7: static char * g_szStdCall = "__stdcall" ; 8: static char * g_szCdeclCall = "__cdecl" ; 9: static char * g_szFastCall = "__fastcall" ; 10: static char * g_szNakedCall = "__naked" ; 11: 12: // The extern "C" turns off all C++ name decoration. 13: extern "C" 14: { 15: 16: // The __cdecl function 17: void CDeclFunction ( char * szString , 18: unsigned long ulLong , 19: char chChar ) ; 20: 21: // The __stdcall function 22: void __stdcall StdCallFunction ( char * szString , 23: unsigned long ulLong , 24: char chChar ) ; 25: // The __fastcall function 26: void __fastcall FastCallFunction ( char * szString , 27: unsigned long ulLong , 28: char chChar ) ; 29: 30: // The naked function. The declspec goes on the definition, not on 31: // the declaration. 32: int NakedCallFunction ( char * szString , 33: unsigned long ulLong , 34: char chChar ) ; 35: } 36: 37: void main ( void ) 38: { 00401000 55 push ebp 00401001 8B EC mov ebp,esp 00401003 53 push ebx 00401004 56 push esi 00401005 57 push edi 39: // Call each function to generate the code. I separate each 40: // function with a couple of NOP bytes to make it easier to read 41: // the disassembly. 42: __asm NOP __asm NOP 00401006 90 nop 00401007 90 nop 43: CDeclFunction ( g_szCdeclCall , 1 , 'a' ) ; 00401008 6A 61 push 61h 0040100A 6A 01 push 1 0040100C A1 14 30 40 00 mov eax,[g_szCdeclCall (00403014)] 00401011 50 push eax 00401012 E8 45 00 00 00 call CDeclFunction (0040105c) 00401017 83 C4 0C add esp,0Ch 44: __asm NOP __asm NOP 0040101A 90 nop 0040101B 90 nop 45: StdCallFunction ( g_szStdCall , 2 , 'b' ) ; 0040101C 6A 62 push 62h 0040101E 6A 02 push 2 00401020 8B 0D 10 30 40 00 mov ecx,dword ptr [g_szStdCall (00403010)] 00401026 51 push ecx 00401027 E8 3D 00 00 00 call StdCallFunction (00401069) 46: __asm NOP __asm NOP 0040102C 90 nop 0040102D 90 nop 47: FastCallFunction ( g_szFastCall , 3 , 'c' ) ; 0040102E 6A 63 push 63h 00401030 BA 03 00 00 00 mov edx,3 00401035 8B 0D 18 30 40 00 mov ecx,dword ptr [g_szFastCall (00403018)] 0040103B E8 38 00 00 00 call FastCallFunction (00401078) 48: __asm NOP __asm NOP 00401040 90 nop 00401041 90 nop 49: NakedCallFunction ( g_szNakedCall , 4 , 'd' ) ; 00401042 6A 64 push 64h 00401044 6A 04 push 4 00401046 8B 15 1C 30 40 00 mov edx,dword ptr [g_szNakedCall (0040301c)] 0040104C 52 push edx 0040104D E8 40 00 00 00 call NakedCallFunction (00401092) 00401052 83 C4 0C add esp,0Ch 50: __asm NOP __asm NOP 00401055 90 nop 00401056 90 nop 51: 52: } 00401057 5F pop edi 00401058 5E pop esi 00401059 5B pop ebx 0040105A 5D pop ebp 0040105B C3 ret 53: 54: void CDeclFunction ( char * szString , 55: unsigned long ulLong , 56: char chChar ) 57: { 0040105C 55 push ebp 0040105D 8B EC mov ebp,esp 0040105F 53 push ebx 00401060 56 push esi 00401061 57 push edi 58: __asm NOP __asm NOP 00401062 90 nop 00401063 90 nop 59: } 00401064 5F pop edi 00401065 5E pop esi 00401066 5B pop ebx 00401067 5D pop ebp 00401068 C3 ret 60: 61: void __stdcall StdCallFunction ( char * szString , 62: unsigned long ulLong , 63: char chChar ) 64: { 00401069 55 push ebp 0040106A 8B EC mov ebp,esp 0040106C 53 push ebx 0040106D 56 push esi 0040106E 57 push edi 65: __asm NOP __asm NOP 0040106F 90 nop 00401070 90 nop 66: } 00401071 5F pop edi 00401072 5E pop esi 00401073 5B pop ebx 00401074 5D pop ebp 00401075 C2 0C 00 ret 0Ch 67: 68: void __fastcall FastCallFunction ( char * szString , 69: unsigned long ulLong , 70: char chChar ) 71: { 00401078 55 push ebp 00401079 8B EC mov ebp,esp 0040107B 83 EC 08 sub esp,8 0040107E 53 push ebx 0040107F 56 push esi 00401080 57 push edi 00401081 89 55 F8 mov dword ptr [ebp-8],edx 00401084 89 4D FC mov dword ptr [ebp-4],ecx 72: __asm NOP __asm NOP 00401087 90 nop 00401088 90 nop 73: } 00401089 5F pop edi 0040108A 5E pop esi 0040108B 5B pop ebx 0040108C 8B E5 mov esp,ebp 0040108E 5D pop ebp 0040108F C2 04 00 ret 4 74: 75: __declspec(naked) int NakedCallFunction ( char * szString , 76: unsigned long ulLong , 77: char chChar ) 78: { 00401092 90 nop 00401093 90 nop 79: __asm NOP __asm NOP 80: // Naked functions must explicitly do a return. 81: __asm RET 00401094 C3 ret