Platform Invoke

Whether a developer is trying to call a library of his existing unmanaged code, accessing unmanaged code in the operating system not exposed in any managed API, or trying to achieve maximum performance for a particular algorithm that performs faster by avoiding the runtime overhead of type checking and garbage collection, at some point she must call into unmanaged code. The CLI provides this capability through P/Invoke. With P/Invoke, you can make API calls into exported functions of unmanaged DLLs.

All of the APIs invoked in this section are Windows APIs. Although the same APIs are not available on other platforms, developers can still use P/Invoke for APIs native to their platform, or for calls into their own DLLs. The guidelines and syntax are the same.

Declaring External Functions

Once the target function is identified, the next step of P/Invoke is to declare the function with managed code. Just like all regular methods that belong to a class, you need to declare the targeted API within the context of a class, but by using the extern modifier. Listing 17.1 demonstrates how to do this.

Listing 17.1. Declaring an External Method

using System; using System.Runtime.InteropServices; class VirtualMemoryManager {    [DllImport("kernel32.dll", EntryPoint="GetCurrentProcess")]    internal static extern IntPtr GetCurrentProcessHandle(); }

In this case, the class is VirtualMemoryManager, because it will contain functions associated with managing memory. (This particular function is available directly off the System.Diagnostics.Processor class, so there is no need to declare it in real code.)

extern methods are always static and don't include any implementation. Instead, the DllImport attribute, which accompanies the method declaration, points to the implementation. At a minimum, the attribute needs the name of the DLL that defines the function. The runtime determines the function name from the method name. However, it is possible to override this default using the EnTRyPoint named parameter to provide the function name. (The .NET platform will automatically attempt calls to the Unicode [...W] or ASCII [...A] API version.)

It this case, the external function, GetCurrentProcess(), retrieves a pseudohandle for the current process which you will use in the call for virtual memory allocation. Here's the unmanaged declaration:

HANDLE GetCurrentProcess();

Parameter Data Types

Assuming the developer has identified the targeted DLL and exported function, the most difficult step is identifying or creating the managed data types that correspond to the unmanaged types in the external function. ^[1]Listing 17.2 shows a more difficult API.

^[1] One particularly helpful resource for declaring Win32 APIs is www.pinvoke.net. This provides a great starting point for many APIs, helping to avoid some of the subtle problems that can arise when coding an external API call from scratch.

Listing 17.2. The VirtualAllocEx() API

LPVOID VirtualAllocEx( HANDLE hProcess,       // The handle to a process. The                        // function allocates memory within                        // the virtual address space of this                        // process. LPVOID lpAddress,      // The pointer that specifies a                        // desired starting address for the                        // region of pages that you want to                        // allocate. If lpAddress is NULL,                        // the function determines where to                        // allocate the region. SIZE_T dwSize,         // The size of the region of memory to                        // allocate, in bytes. If lpAddress                        // is NULL, the function rounds dwSize                        // up to the next page boundary. DWORD flAllocationType,// The type of memory allocation. DWORD flProtect);      // The type of memory allocation.

VirtualAllocEx() allocates virtual memory that the operating system specifically designates for execution or data. To call it, you also need corresponding definitions in managed code for each data type; although common in Win32 programming, HANDLE, LPVOID, SIZE_T, and DWORD are undefined in the CLI managed code. The declaration in C# for VirtualAllocEx(), therefore, is shown in Listing 17.3.

Listing 17.3. Declaring the VirtualAllocEx() API in C#

 using System; using System.Runtime.InteropServices; class VirtualMemoryManager {   [DllImport("kernel32.dll")]   internal static extern IntPtr GetCurrentProcess();   [DllImport("kernel32.dll", SetLastError = true)]   private static extern IntPtr VirtualAllocEx(        IntPtr hProcess,        IntPtr lpAddress,        IntPtr dwSize,        AllocationType flAllocationType,        uint flProtect); }

One distinct characteristic of managed code is the fact that primitive data types such as int do not change size based on the processor. Whether 16, 32, or 64 bits, int is always 32 bits. In unmanaged code, however, memory pointers will vary depending on the processor. Therefore, instead of mapping types such as HANDLE and LPVOID simply to ints, you need to map to System.IntPtr, whose size will vary depending on the processor memory layout. This example also uses an AllocationType enum, which I discuss in the section Simplifying API Calls with Wrappers, later in this chapter.

Using ref Rather Than Pointers

Frequently, unmanaged code uses pointers for pass-by-reference parameters. In these cases, P/Invoke doesn't require that you map the data type to a pointer in managed code. Instead, you map the corresponding parameters to ref (or out), depending on whether the parameter is in-out or just out. In Listing 17.4, lpflOldProtect, whose data type is PDWORD, is an example that returns the "pointer to a variable that receives the previous access protection of the first page in the specified region of pages."

Listing 17.4. Using ref and out Rather Than Pointers

class VirtualMemoryManager {   // ...   [DllImport("kernel32.dll", SetLastError = true)]   static extern bool VirtualProtectEx(       IntPtr hProcess, IntPtr lpAddress,       IntPtr dwSize, uint flNewProtect,       ref uint lpflOldProtect); }

In spite of the fact that lpflOldProtect is documented as [out], the description goes on to mention that the parameter must point to a valid variable and not NULL. The inconsistency is confusing, but common. The guideline is to use ref rather than out for P/Invoke type parameters since the callee can always ignore the data passed with ref, but the converse will not necessarily succeed.

The other parameters are virtually the same as VirtualAllocEx(), except that the lpAddress is the address returned from VirtualAllocEx(). In addition, flNewProtect specifies the exact type of memory protection: page execute, page read-only, and so on.

Using StructLayoutAttribute for Sequential Layout

Some APIs involve types that have no corresponding managed type. To call these requires redeclaration of the type in managed code. You declare the unmanaged COLOREF struct, for example, in managed code (see Listing 17.5).

Listing 17.5. Declaring Types from Unmanaged Structs

 [StructLayout(LayoutKind.Sequential)] struct ColorRef {    public byte Red;    public byte Green;    public byte Blue;    // Turn off warning about not accessing Unused.    #pragma warning disable 414    private byte Unused;    #pragma warning restore 414    public ColorRef(byte red, byte green, byte blue)    {          Blue = blue;          Green = green;          Red = red;          Unused = 0;    } }

Various Microsoft Windows color APIs use COLORREF to represent RGB colors (levels of red, green, and blue).

The key in this declaration is StructLayoutAttribute. By default, managed code can optimize the memory layouts of types, so layouts may not be sequential from one field to the next. To force sequential layouts so that a type maps directly and can be copied bit for bit (blitted) from managed to unmanaged code and vice versa, you add the StructLayoutAttribute with the LayoutKind.Sequential enum value. (This is also useful when writing data to and from filestreams where a sequential layout may be expected.)

Since the unmanaged (C++) definition for struct does not map to the C# definition, there is not a direct mapping of unmanaged struct to managed struct. Instead, developers should follow the usual C# guidelines about whether the type should behave like a value or a reference type, and whether the size is small (approximately less than 16 bytes).

Error Handling

One inconvenient characteristic of Win32 API programming is the fact that it frequently reports errors in inconsistent ways. For example, some APIs return a value (0, 1, false, and so on) to indicate an error, and others set an out parameter in some way. Furthermore, the details of what went wrong require additional calls to the GetLastError() API and then an additional call to FormatMessage() to retrieve an error message corresponding to the error. In summary, Win32 error reporting in unmanaged code seldom occurs via exceptions.

Fortunately, the P/Invoke designers provided a mechanism for handling this. To enable this, given the SetLastError named parameter of the DllImport attribute is true, it is possible to instantiate a System .ComponentModel.Win32Exception() that is automatically initialized with the Win32 error data immediately following the P/Invoke call (see Listing 17.6).

Listing 17.6. Win32 Error Handling

 class VirtualMemoryManager {    [DllImport("kernel32.dll", ", SetLastError = true)]    private static extern IntPtr VirtualAllocEx(        IntPtr hProcess,        IntPtr lpAddress,        IntPtr dwSize,        AllocationType flAllocationType,        uint flProtect);        // ...    [DllImport("kernel32.dll", SetLastError = true)]    static extern bool VirtualProtectEx(        IntPtr hProcess, IntPtr lpAddress,        IntPtr dwSize, uint flNewProtect,        refuint lpflOldProtect);    [Flags]    private enum AllocationType : uint    {        // ...    }    [Flags]    private enum ProtectionOptions    {        // ...    }    [Flags]    private enum MemoryFreeType    {        // ...    }    public static IntPtr AllocExecutionBlock(         int size, IntPtr hProcess)    {         IntPtr codeBytesPtr;         codeBytesPtr = VirtualAllocEx(             hProcess, IntPtr.Zero,             (IntPtr)size,             AllocationType.Reserve | AllocationType.Commit,             (uint)ProtectionOptions.PageExecuteReadWrite);         if (codeBytesPtr == IntPtr.Zero)         {             throw new System.ComponentModel.Win32Exception();                                  }         uint lpflOldProtect = 0;         if (!VirtualProtectEx(             hProcess, codeBytesPtr,             (IntPtr)size,             (uint)ProtectionOptions.PageExecuteReadWrite,             ref lpflOldProtect))         {             throw new System.ComponentModel.Win32Exception();                                  }         return codeBytesPtr;    }        public static IntPtr AllocExecutionBlock(int size)        {        return AllocExecutionBlock(               size, GetCurrentProcessHandle());        }    }

This enables developers to provide the custom error checking that each API uses while still reporting the error in a standard manner.

Listing 17.1 and Listing 17.3 declared the P/Invoke methods as internal or private. Except for the simplest of APIs, wrapping methods in public wrappers that reduce the complexity of the P/Invoke API calls is a good guideline that increases API usability and moves toward object-oriented type structure. The AllocExecutionBlock() declaration in Listing 17.6 provides a good example of this.

Using SafeHandle

Frequently, P/Invoke involves a resource, such as a window handle, that code needs to clean up after using it. Instead of requiring developers to remember this and manually code it each time, it is helpful to provide a class that implements IDisposable and a finalizer. In Listing 17.7, for example, the address returned after VirtualAllocEx() and VirtualProtectEx() requires a follow-up call to VirtualFreeEx(). To provide built-in support for this, you define a VirtualMemoryPtr class that derives from System.Runtime.InteropServices.SafeHandle (this is new in .NET 2.0).

Listing 17.7. Managed Resources Using SafeHandle

     public classVirtualMemoryPtr :   System.Runtime.InteropServices.SafeHandle   {     public VirtualMemoryPtr(int memorySize) :          base(IntPtr.Zero, true)     {          ProcessHandle =              VirtualMemoryManager.GetCurrentProcessHandle();          MemorySize = (IntPtr)memorySize;          AllocatedPointer =              VirtualMemoryManager.AllocExecutionBlock(              memorySize, ProcessHandle);              Disposed = false;     }     public readonly IntPtr AllocatedPointer;     readonly IntPtr ProcessHandle;     readonly IntPtr MemorySize;     bool Disposed;     public static implicit operator IntPtr(           VirtualMemoryPtr virtualMemoryPointer)     {           return virtualMemoryPointer.AllocatedPointer;     }     // SafeHandle abstract member      public override boolIsInvalid     {            get            {                    return Disposed;            }     }     // SafeHandle abstract member     protected override bool ReleaseHandle()     {            if (!Disposed)            {                   Disposed = true;                   GC.SuppressFinalize(this);                   VirtualMemoryManager.VirtualFreeEx(ProcessHandle,                       AllocatedPointer, MemorySize);            }            return true;    } }

System.Runtime.InteropServices.SafeHandle includes the abstract members IsInvalid and ReleaseHandle(). In the latter, you place your cleanup code; the former indicates whether the cleanup code has executed yet.

With VirtualMemoryPtr, you can allocate memory simply by instantiating the type and specifying the needed memory allocation.

Advanced Topic: Using IDisposable Explicitly in Place of SafeHandle

In C# 1.0, System.Runtime.InteropServices.SafeHandle is not available. Instead, a custom implementation of IDisposable, as shown in Listing 17.8, is necessary.

Listing 17.8. Managed Resources without SafeHandle but Using IDisposable

public struct VirtualMemoryPtr : IDisposable {    public VirtualMemoryPtr(int memorySize)    {       ProcessHandle =               VirtualMemoryManager.GetCurrentProcessHandle();       MemorySize = (IntPtr)memorySize;       AllocatedPointer =               VirtualMemoryManager.AllocExecutionBlock(               memorySize, ProcessHandle);       Disposed = false;    }    public readonly IntPtr AllocatedPointer;    readonly IntPtr ProcessHandle;    readonly IntPtr MemorySize;    bool Disposed;    public static implicit operator IntPtr(       VirtualMemoryPtr virtualMemoryPointer)    {       return virtualMemoryPointer.AllocatedPointer;    }    #region IDisposable Members    public void Dispose()    {       if (!Disposed)       {               Disposed = true;               GC.SuppressFinalize(this);               VirtualMemoryManager.VirtualFreeEx(ProcessHandle,                    AllocatedPointer, MemorySize);        }    }    #endregion }

In order for VirtualMemoryPtr to behave with value type semantics, you need to implement it as a struct. However, the consequence of this is there can be no finalizer, since value types are not managed by the garbage collector. This means the developer using the type must remember to clean up the code. There is no fallback mechanism if he doesn't.

The second restriction is not to pass or copy the instance outside the method. This is a common guideline of IDisposable implementing types. Their scope should be left within a using statement and they should not be passed as parameters to other methods that could potentially save them beyond the life of the using scope.

Calling External Functions

Once you declare the P/Invoke functions, you invoke them just as you would any other class member. The key, however, is that the imported DLL must be in the path, including the executable directory, so that it can be successfully loaded. Listing 17.6 and Listing 17.7 provide demonstrations of this. However, they rely on some constants.

Since flAllocationType and flProtect are flags, it is a good practice to provide constants or enums for each. Instead of expecting the caller to define these, encapsulation suggests you provide them as part of the API declaration, as shown in Listing 17.9.

Listing 17.9. Encapsulating the APIs Together

 class VirtualMemoryManager {       // ...      /// <summary>      /// The type of memory allocation. This parameter must      /// contain one of the following values.      /// </summary>      [Flags]      private enum AllocationType : uint      {              /// <summary>              /// Allocates physical storage in memory or in the              /// paging file on disk for the specified reserved              /// memory pages. The function initializes the memory              /// to zero.              /// </summary> Commit = 0x1000,               /// <summary>              /// Reserves a range of the process's virtual address               /// space without allocating any actual physical              /// storage in memory or in the paging file on disk.               /// </summary>              Reserve = 0x2000,              /// <summary>              /// Indicates that data in the memory range specified by              /// lpAddress and dwSize is no longer of interest. The              /// pages should not be read from or written to the              /// paging file. However, the memory block will be used              /// again later, so it should not be decommitted. This              /// value cannot be used with any other value.              /// </summary>              Reset = 0x80000,              /// <summary>              /// Allocates physical memory with read-write access.              /// This value is solely for use with Address Windowing              /// Extensions (AWE) memory.              /// </summary>              Physical = 0x400000,               /// <summary>              /// Allocates memory at the highest possible address.               /// </summary>              TopDown = 0x100000,        }          /// <summary>          /// The memory protection for the region of pages to be          /// allocated.          /// </summary>          [Flags]          private enum ProtectionOptions : uint          {              /// <summary>              /// Enables execute access to the committed region of              /// pages. An attempt to read or write to the committed              /// region results in an access violation.              /// </summary>              Execute = 0x10,              /// <summary>              /// Enables execute and read access to the committed              /// region of pages. An attempt to write to the               /// committed region results in an access violation.              /// </summary>              PageExecuteRead = 0x20,              /// <summary>              /// Enables execute, read, and write access to the              /// committed region of pages.              /// </summary>              PageExecuteReadWrite = 0x40,              // ...           }           /// <summary>           /// The type of free operation           /// </summary>           [Flags]           private enum MemoryFreeType : uint           {              /// <summary>              /// Decommits the specified region of committed pages.              /// After the operation, the pages are in the reserved              /// state.              /// </summary>              Decommit = 0x4000,              /// <summary>              /// Releases the specified region of pages. After this              /// operation, the pages are in the free state.              /// </summary>              Release = 0x8000          }           // ... }

The advantage of enums is that they group together each value. Furthermore, they limit the scope to nothing else besides these values.

Simplifying API Calls with Wrappers

Whether it is error handling, structs, or constant values, one goal of good API developers is to provide a simplified managed API that wraps the underlying Win32 API. For example, Listing 17.10 overloads VirtualFreeEx() with public versions that simplify the call.

Listing 17.10. Wrapping the Underlying API

   class VirtualMemoryManager    {          // ...         [DllImport("kernel32.dll", SetLastError = true)]         static extern bool VirtualFreeEx(                 IntPtr hProcess, IntPtr lpAddress,                 IntPtr dwSize, IntPtr dwFreeType);         public static bool VirtualFreeEx(                 IntPtr hProcess, IntPtr lpAddress,                 IntPtr dwSize)          {                 bool result = VirtualFreeEx(                         hProcess, lpAddress, dwSize,                         (IntPtr)MemoryFreeType.Decommit);                 if (!result)                 {                         throw new System.ComponentModel.Win32Exception();                 }                 return result;           }           public static bool VirtualFreeEx(                 IntPtr lpAddress, IntPtr dwSize)                 {                         return VirtualFreeEx(                         GetCurrentProcessHandle(), lpAddress, dwSize);                  }                  [DllImport("kernel32", SetLastError = true)]                  static extern IntPtr VirtualAllocEx(                  IntPtr hProcess,                  IntPtr lpAddress,                  IntPtr dwSize,                  AllocationType flAllocationType,                  uint flProtect);           // ...  }

Function Pointers Map to Delegates

One last P/Invoke key is that function pointers in unmanaged code map to delegates in managed code. To set up a Microsoft Windows timer, for example, you would provide a function pointer that the timer could call back on, once it had expired. Specifically, you would pass a delegate instance that matched the signature of the callback.

Guidelines

Given the idiosyncrasies of P/Invoke, there are several guidelines to aid in the process of writing such code.

• Check that no managed classes already expose the APIs.

• Define API external methods as private or, in simple cases, internal.

• Provide public wrapper methods around the external methods that handle the data type conversions and error handling.

• Overload the wrapper methods and provide a reduced number of required parameters by inserting defaults for the extern method call.

• Use enum or const to provide constant values for the API as part of the API's declaration.

• For all P/Invoke methods that support GetLastError(), be sure to assign the SetLastError named attribute to TRue. This allows the reporting of errors via System.ComponentModel.Win32Exception.

• Wrap resources, such as handles, into classes that derive from System.Runtime.InteropServices.SafeHandle or that support IDisposable.

• Function pointers in unmanaged code map to delegate instances in managed code. Generally, this requires the declaration of a specific delegate type that matches the signature of the unmanaged function pointer.

• Map input/output and output parameters to ref parameters instead of relying on pointers.

The last bullet implies C#'s support for pointers, described in the next section.