Pointers, Handles, and Resources

Regardless of the legacy or unmanaged technology you're working with, there are a set of common challenges that you will face. This section introduces these topics and mentions briefly some helpful hints and useful ways to think about the problem. Most of the detailed "how to"—style material can be found later in this chapter.

"Interoperability" Defined

Interoperability is a simple concept: The ability to invoke code and/or share data with programs or libraries across heterogeneous technologies. That's admittedly a broad definition. Pragmatically, what does that actually mean? As an illustration, consider that you've written a large Win32 DLL in unmanaged C++. In other words, it uses the C++ language, relies on functions exposed by Win32 DLLs, and was compiled down to the Win32 PE/COFF file format as a standard C++ DLL. This code might have taken years to develop and been maintained for even longer. If you suddenly need to write a new business component, what do you do? Extend the existing codebase, using the same technologies as before? Or do you try to incrementally adopt the .NET Framework?

It turns out that you have a few options in this scenario, ranging from recompiling with VC++ and targeting the CLR, in which case you can then mix managed and unmanaged functions to your existing codebase — still using C++ syntax — effectively adopting managed code incrementally. Alternatively, you could write a new managed DLL in C# and link with your existing C++ DLL. The first option speaks to the power of the new VC++ (i.e., MC++ in 1.x and C++/CLI in 2.0), namely that it has unmanaged and managed code interoperability built right into the language. The latter scenario is another form of interoperability, although it results in more moving parts: two separate DLLs, one of which knows how to call into the other. This can often be a cleaner architecture to maintain.

Interoperability is simple to understand but can be challenging in practice. This section introduces a few of the common concepts you will encounter, but by no means is it the extent of it. If your goal is to avoid touching your legacy code altogether—indeed, sometimes this is the only option, for example for purchased components, lost source code, or a bit-rotted build infrastructure that is too costly to rebuild — it can quite often be a frustrating undertaking.

Native Pointers in the CTS (IntPtr)

The IntPtr class (native int in IL) and its closely related cousin UIntPtr (unsigned native int in IL) are thin wrappers on top of an unmanaged pointer or handle — stored internally as a void*. IntPtr serves the following primary purposes:

• The size of the pointer is abstracted from the programmer. The size of the actual underlying pointer is the native pointer size on the runtime platform. This means 32-bits (DWORD) on 32-bit platforms (x86), and 64 bits (quad-WORD) on 64-bit platforms (AMD-64, IA-64). As is the case with unmanaged pointers, you get the benefit of a larger addressable virtual memory space for free when you upgrade to a new platform. Accessing the static property IntPtr.Size tells you the runtime size of an IntPtr.

• It enables managed code to pass around pointers without requiring recipients to declare themselves as unsafe, using the special type system support IntPtrs have. Unwrapping the IntPtr to access the raw pointer value is done automatically as part of the managed-to-unmanaged transition. If code were to work with void*s, it would immediately be tagged as unsafe by compilers and the runtime, requiring a certain level of trust in CAS-relevant execution contexts. IntPtr is a lightweight wrapper that eliminates this need.

IntPtrs are often used to represent offsets into a process's virtual memory address space. For unmanaged code written in say C++, this is the most common method with which to share objects inside the same process. A pointer is just an integer value that, when dereferenced, enables you to manipulate the bits stored in memory at that address. Unfortunately, working with IntPtrs by hand is much like working with pointers in the unmanaged world; that is, you have to worry about leaks, dangling pointers, recycling, and so forth. SafeHandle helps to hide many of these gruesome details. SafeHandle is to shared_ptr<void*> as IntPtr is to void*.

This example shows raw manipulation of an underlying pointer passed around via an IntPtr:

 using System; class Program {     unsafe static void Main()     {         int a = 10;         IntPtr ip = new IntPtr(&a);         SetByte(ip, 1, 1);     }     static unsafe void SetByte(IntPtr base, int offset, byte value)     {         if (base == IntPtr.Zero)             throw new ArgumentOutOfRangeException();         byte* pTarget = (byte*)base.ToPointer() + offset;         *pTarget = value;     } } 

While this is terribly useful, we see a few interesting things here. Using C#'s unsafe constructs, we can get the address of a value on the stack using the address-of operator, for example &a. We then pass that to IntPtr's constructor, which expects a void*. Then we call our own custom function that sets some specific byte at an offset based on the pointer using the IntPtr.ToPointer function to retrieve and work with the raw pointer.

Alternatively, an IntPtr can be an OS handle. For example, P/Invoke signatures use IntPtr instead of HANDLE. The term handle is scarier than it sounds. On Windows, it's just an integer that is used to index into a per-process handle table, each handle of which references an OS-managed resource. For example, each kernel object (e.g., process, thread, mutex) is accessed via its handle, passed to the Win32 functions that manipulate it. A file object, for instance, also has a unique handle that gets allocated upon opening a file. All code in the process uses this handle to work with Win32 I/O functions. When clients are done with a resource, they close the handle. This instructs the OS to relinquish resources associated with the handle (e.g., unlocking the file for write access).

Handle Recycling

It turns out that IntPtrs are susceptible to a security problem called handle recycling. To understand how a handle recycling attack can be mounted, you have to be pretty familiar with how thread races can occur. We discussed race conditions in detail in the threading chapter, Chapter 10. The trick here is to get multiple threads simultaneously accessing a single object that encapsulates a handle. One tries to get the managed wrapper class to close the handle, while the other simultaneously initiates an operation on the instance that attempts to use the handle, for example by passing it to a Win32 function.

Because IntPtr doesn't do any sort of reference counting, if one thread says it is done with the IntPtr and closes the handle (e.g., via CloseHandle), another thread could come in and load an IntPtr onto its stack just before the call to CloseHandle. Then the thread that already has the IntPtr on its stack would be working with a dangling handle at that point, and the ensuing operation that tried to use it would see unpredictable behavior at best and a security hole at worst.

Consider an example:

 using System; class MyFile : IDisposable {     private IntPtr hFileHandle;     public MyFile(string filename)     {         hFileHandle = OpenFile(filename, out ofStr, OF_READWRITE);     }     ~MyFile()     {         Dispose();     }     public void Dispose()     {         if (hFileHandle != IntPtr.Zero)             CloseHandle(hFileHandle);         hFileHandle = IntPtr.Zero;         GC.SuppressFinalize(this);     }     public int ReadBytes(byte[] buffer)     {         // First, ensure the file's not closed         IntPtr hFile = hFileHandle;         if (hFile == IntPtr.Zero)             throw new ObjectDisposedException();         uint read;         if (!ReadFile(hFile, buffer,                 buffer.Length, out read, IntPtr.Zero))             throw new Exception("Error "+ Marshal.GetLastWin32Error());         return read;     }     private const OF_READWRITE = 0x00000002;     /* P/Invoke signatures for these functions omitted for brevity:            Kernel32!OpenFile            Kernel32!ReadHandle            Kernel32!CloseHandle     */ } 

All we need to do is to have a situation where somebody is calling Dispose while another thread calls ReadBytes on the same instance. This could lead to: (1) ReadBytes begins running, loads the value for hFileHandle onto its stack (remember: it's a value type); (2) Dispose is scheduled for execution, either preempting ReadBytes (on a uniprocessor) or perhaps running in parallel (on a multiprocessor); (3) Dispose executes completely, closing the handle and setting hFileHandle to IntPtr.Zero; (4) ReadBytes still has the old value on its stack and passes it to ReadFile! Oops!

This is a great case study in the pervasiveness and hard-to-detect nature of races.

Note

Notice that these problems are the same style of problem you tend to encounter in unmanaged code. Dangling pointers aren't anything new, but managed code was supposed to shield us from all of these problems! It turns out that we've developed new abstractions to help manage this, but a solid understanding of how pointers and handles function is still crucial for those who venture into the world of unmanaged interoperability.

This situation mentioned above could, in fact, be disastrous. If Windows happened to reuse the same integer value as the previous IntPtr (which it does from time to time), malicious code could see a resource owned by somebody else in the process. This is more likely than you might think. In this example, code might try to perform operations on the new handle thinking it was still working with the old one. For example, it could enable the program to access a file handle opened by another program on the machine or scribble on somebody else's shared data (possible corrupting it).

If all of this sounds scary, it is. CAS would be entirely subverted. Windows kernel object ACLs might help catch this problem — assuming that handles were opened using them (see Chapter 9 for details) — but very few programs use ACLs correctly, especially in managed code (not the least of which is due to poor System.IO support prior to 2.0). Refer to the SafeHandle class described just below for details on how this problem has been solved. I already gave a hint (reference counting).

Nonpointer Usage

It turns out the IntPtr is a convenient mechanism to store field data that naturally grows with the memory addressability capabilities of the platform beneath your program. For example, arrays on the CLR use IntPtrs for their internal length field because an array's capacity naturally grows with some proportion to the increased addressable memory space. While this enables arrays to hold more than enough elements, what you ideally want is a data type that is able to store sizeof(IntPtr) / sizeof(<element_type>) bytes (assuming that your computer's RAM holds nothing but the data in your array) — but the size of IntPtr is the dominating factor in this calculation and therefore is significantly more accurate than, say, an Int32.

Memory and Resource Management

When a managed object holds on to an unmanaged resource, it must explicitly release that resource once it is done using it. Otherwise, your application will leak memory or resource handles. Unless you are writing libraries or applications that interoperate with unmanaged code, you seldom have to worry about these details. The farthest you must go is remembering to call Dispose (and even without that, a type's Finalize method will usually clean up for you).

For leaks that are process-wide, you will often observe a growing working set or handle count that can eventually lead to out-of-memory conditions. Alternatively, your program's performance will degrade, slowly grinding to a halt (e.g., thrashing the disk due to paging), and the user will notice and respond by hitting End Task in Task Manager. Once the process is shut down, the OS can reclaim the memory. For system-wide resource leaks, the situation isn't so easy. The OS can't reclaim such resources at process shutdown time. Resources can be orphaned, interfering with other programs on the machine, or memory usage might just leak, in some cases requiring that the user reboot the machine. Clearly, you want to avoid these situations at all costs.

Note

ASP.NET was designed to be tolerant of memory leaks such as this. Most sophisticated hosts try to be resilient in their own special way. Policies differ of course. ASP.NET recycles processes that leak memory. It provides configurable thresholds for memory utilization; if your process exceeds it, the host shuts down and restarts the worker process. SQL Server, on the other hand, tries very hard to shut down code in a way that prevents resource leaks. Critical finalization — a topic discussed later on — was developed to assist it in achieving this goal.

Example: Using Unmanaged Memory

First, let's take a look at a simple example of this challenge. Well, it will start simply, and we will then incrementally make it more complex. We'll go from very leaky code to leak tolerant code in no time. Imagine that you are allocating some block of unmanaged memory for the lifetime of a single function:

 void MyFunction {     IntPtr ptr = Marshal.AllocHGlobal(1024);     // Do something with 'ptr'...     Marshal.FreeHGlobal(ptr); } 

The Marshal class can be found in the System.Runtime.InteropServices namespace. The AllocHGlobal function allocates a chunk of unmanaged memory inside the current process on the unmanaged heap and returns an IntPtr pointing at it. It's the logical equivalent to malloc. And much like malloc, unless you explicitly free the memory at the returned address, the newly allocated block will remain used and will never get reclaimed. Freeing is done via the FreeHGlobal method.

Unfortunately, this code is about as bad as it gets (aside from perhaps forgetting the FreeHGlobal altogether). If you run code in between the allocation and free which can throw an exception, you will never free the memory block. Furthermore, an asynchronous exception (like a ThreadAbortException) can occur anywhere, including between the call to AllocHGlobal and the store to our ptr stack variable. If the caller of MyFunction catches the exception and decides to continue executing the program, you've got a leak! The IntPtr is entirely lost, and you have no hook using which to deallocate the memory. It won't go away until you shut down the process.

If you have lots of calls to MyFunction-like code throughout your program, you could end up eating up all of the available memory on your user's machine very quickly. Or, in ASP.NET applications, you'll trigger a recycle more frequently than you'd probably like.

Cleaning Up with Try/Finally

A partial solution is very easy. Just use try/finally blocks to guarantee the release of the memory:

 void MyFunction {     IntPtr ptr = Marshal.AllocHGlobal(1024);     try     {         // Do something with ‘ptr'...     }     finally     {         if (ptr != IntPtr.Zero)         {             Marshal.FreeHGlobal(ptr);             ptr = IntPtr.Zero;         }     } } 

Notice that we set the ptr to IntPtr.Zero after freeing it. This is certainly a discretionary action. But it helps for debuggability; if we accidentally passed the pointer to some other function, any attempted dereferences will generate access violations. Access violations (a.k.a. AVs) are generally much easier to debug than memory corruption.

(The above pattern is not foolproof for the asynchronous exception reason mentioned briefly. We'll see how to fix that momentarily.)

Object Lifetime != Resource Lifetime

Resources are everywhere in the Framework. OS handles can refer to open files, database connections, GUI elements, and so forth. Many of your favorite system types wrap handles and hide them from you as a user. For example, each FileStream contains a file handle, SqlConnections rely on database connections to SQL Server databases, and Controls often contain HWNDs pointing to UI element data structures, for example. Such wrapper types offer implementations of IDisposable for ease of use. Simply wrapping such types in C# or VB's using statement facilities eager cleanup:

 using (FileStream fs = /*...*/) {     // Do something with ‘fs'... } // ‘fs' was automatically cleaned up at the end of the block 

Now consider what happens when a managed object owns the lifetime of such a resource. Ideally, it will offer users a way to deterministically clean up resources when they know for sure they're done with them. But in the worst case, it must clean up resources prior to being reclaimed by the GC.

Using the first example as a starting point, what if our own custom type MyResourceManager had an IntPtr as a field?

 class MyResourceManager {     private IntPtr ptr;     public MyResourceManager()     {         ptr = Marshal.AllocHGlobal(1024);     } } 

In this case, when and where does it make sense to deallocate the memory that ptr refers to after object construction? We know that managed object lifetimes are controlled by the GC. We can use that as a starting point to answer this question. In fact, you've seen the building blocks already: Chapter 3 for the internals of GC and Chapter 5 specifically for the feature we're about to discuss.

Finalization (Lazy Cleanup)

Finalizers enables an object to run additional code when it is collected by the GC. This is a good "last chance" effort to reclaim any resources associated with an object. (Also consider using either memory pressure or the HandleCollector type — discussed below — to notify the GC that objects are holding on to additional resources. This can help to speed up finalization, and reduce overall system pressure.) If you've been given a handle for a commodity resource, closing it at finalization time is the least you need to do:

 class MyResourceManager {     private IntPtr ptr;     public MyResourceManager()     {         ptr = Marshal.AllocHGlobal(1024);     }     ~MyResourceManager()     {         if (ptr != IntPtr.Zero)         {             Marshal.FreeHGlobal(ptr);             ptr = IntPtr.Zero;         }     } } 

For most resources — for example, file handles, database connections, and scarce COM objects — waiting for the GC to kick in before releasing them back to the system is simply not acceptable. Objects that utilize such resources for longer than a single function call absolutely must provide a deterministic way of getting rid of them.

Disposability (Eager Cleanup)

This is precisely where IDisposable plays a role. If a type implements the IDisposable interface, it indicates that it has control over at least one non-GC resource. A user who invokes the Dispose method asks the object to release any resources in a deterministic predictable manner, rather than having to wait for a type's finalizer to kick in at some indeterminate time in the future. These types must still offer a finalizer in cases where Dispose isn't called, acting as a last chance backstop to prevent resource leaks.

If it's not becoming clear already, Dispose is very much like a destructor in C++: it supplies a deterministic way to release resources. Contrast this with a finalizer that is entirely nondeterministic and is really just a last chance to catch a resource leak. Indeed, C++/CLI 2.0 now uses Dispose as its destructor mechanism. If you write a destructor in C++, it will compile down to a Dispose method, enabling you to utilize the using statement when consuming a C++ type from C#. Furthermore, C++ users can take advantage of stack semantics for IDisposable types and have Dispose called automatically at the end of the scope in which they're used.

Note

It's unfortunate that the C# language designers chose an identical syntax for C# finalizers as C++ uses for destructors (i.e., ~TypeName). This has caused no end of confusion among C# developers, and rightfully so. Hopefully, the difference is clear by now.

After introducing IDisposable, the above class implementation turns into:

 sealed class MyResourceManager : IDisposable {     private IntPtr ptr;     public MyResourceManager()     {         ptr = Marshal.AllocHGlobal(1024);     }     ~MyResourceManager()     {         Dispose(false);     }     public void Dispose()     {         Dispose(true);         GC.SuppressFinalize(this);     }     private void Dispose(bool disposing)     {         if (ptr != IntPtr.Zero)         {             Marshal.FreeHGlobal(ptr);             ptr = IntPtr.Zero;         }     } } 

This implementation enables users of your class to wrap usage in a using block and have the resources automatically deallocated once the end of the block is reached. Notice that the cleanup logic is shared between the Dispose() and ~MyResourceManager (Finalize) methods in the Dispose(bool) method. Because the class was marked as sealed, this is purely an implementation detail. But for subclassing scenarios, it makes hooking into resource cleanup simpler.

Recall our discussion of handle recycling earlier. If you inspect the example above carefully, you will notice that it is prone to this problem. We will see how to fix that below using SafeHandle.

Reliably Managing Resources (SafeHandle)

A new type has been introduced in 2.0 in an attempt to eliminate many of the problems associated with working with raw IntPtrs. It's called SafeHandle and is located in the System.Runtime.InteropServices namespace. SafeHandle is a simple primitive type but is a powerful abstraction. Before diving into the specifics of SafeHandle's members and how to use them, let's review the primary goals of SafeHandle:

• First and foremost, SafeHandle uses a reference counting algorithm to prevent the handle recycling-style attacks that raw IntPtr usage can lead to. It does this by managing the lifetime of a SafeHandle and only closing it once the reference count hits zero. The P/Invoke marshaler takes responsibility for automatically upping this count when a handle crosses an unmanaged boundary and automatically decrements it upon return. Managed APIs are available to manually control the reference count yourself for more sophisticated scenarios. For example, when passing a SafeHandle across threading boundaries, you are generally responsible for incrementing the reference count.

• Ensures reliable release of handles under extreme shutdown conditions via critical finalization (i.e., SafeHandle derives from CriticalFinalizationObject). This is especially important in sophisticated hosting scenarios — such as inside SQL Server — to ensure that resource usage of hosted managed code is adequately protected from leaks. Critical finalization is a complex topic; detailed coverage can be found later in this section.

• Pushing the resource management and finalization burden off of the class author and onto SafeHandle. If you wrap an IntPtr, you are responsible for managing its lifetime. That is, you have to write a Finalize and Dispose method yourself, and deal with the notoriously complex design and implementation issues that come along for the ride. Often you must call GC.KeepAlive to ensure that your finalizer can't execute before the call to a method that used your IntPtr was finished. Now with SafeHandle, you simply write a Dispose method that calls the inner SafeHandle's Dispose, and forego the pain of writing a finalizer altogether. SafeHandle is a tightly encapsulated object with a single purpose in life: reliably manage a single resource.

Now that you have a basic understanding of why SafeHandle was developed, we'll take a quick look at how to use it. There are a few things to note right up front. First, SafeHandle is an abstract class. Most people don't need to worry about writing their own implementations, however, as several are available in the Framework. Second, you'll seldom ever see the actual implementation classes; most of them are internal. And third, you'll rarely ever see a SafeHandle in the open. Most types, such as FileStream, use the handles as private state. Only in cases where you'd normally have been working with a raw IntPtr will you have to directly work with a SafeHandle.

Overview of the SafeHandle API

Before jumping into details about implementing your own SafeHandle class, let's review the public interface of the base type:

 namespace System.Runtime.InteropServices {     public abstract class SafeHandle : CriticalFinalizerObject, IDisposable     {         // Fields         protected IntPtr handle;         // De-/con-structor(s)         protected SafeHandle(IntPtr invalidHandleValue, bool ownsHandle);         protected ~SafeHandle();         // Properties         public bool IsClosed { get; }         public bool IsInvalid { get; }         // Methods         public void Close();         public void DangerousAddRef(ref bool success);         public IntPtr DangerousGetHandle();         public void DangerousRelease();         public void Dispose();         protected void Dispose(bool disposing);         protected bool ReleaseHandle();         protected void SetHandle(IntPtr handle);         public void SetHandleAsInvalid();     } } 

Many of these members are decorated with the ReliabilityContractAttribute (omitted above), found in the System.Runtime.InteropServices.ConstrainedExecution namespace. This is used because many of the functions must execute inside Constrained Execution Regions (CERs) under very tight restrictions. Most of the methods guarantee that they will always succeed and will not corrupt state. Writing code under these constraints is surprisingly difficult. We discuss CERs at a cursory level later in this chapter.

A summary of the available APIs is as follows. Because SafeHandle is an abstract class, there is no public constructor available. Specific implementations may or may not expose one. In fact, many SafeHandle implementations prefer to hide construction behind factories that some other type controls very closely. Regardless of where this happens, the implementation acquires the resource; the call to the base SafeHandle constructor sets the initial internal reference count to 1.

Each SafeHandle exposes the idea of an invalid handle. This is used to detect when the SafeHandle has not been initialized correctly; it may be set by the invalidHandleValue constructor parameter and can be queried by checking the IsInvalid property. IsInvalid returns true if the handle field is equal to the invalidHandleValue. The choice of invalid handle value varies. A set of types exists in the Microsoft .Win32.SafeHandles namespace that encapsulate some of the most common values: SafeHandle ZeroOrMinusOneIsInvalid considers IntPtr.Zero and an IntPtr with a value of -1 to be invalid, while SafeHandleZeroIsInvalid considers just the former to represent an invalid pointer. You might dream up another magic value that makes sense for whatever resource you're interoperating with.

When a client is through using a SafeHandle instance, they will call either Close or Dispose to indicate this. This actually decrements the reference count internally and will only actually release the resource if two conditions hold: the reference count hits 0 and the ownsHandle was true upon construction. This enables easy integration with C# and VB in that you can simply wrap a SafeHandle in a using statement; C++/CLI is even simpler, if you allocate a SafeHandle using stack semantics, it will automatically get disposed when it falls out of scope. IsClosed returns true once the actual underlying resource has been disposed of.

Lastly, there are a set of DangerousXxx functions. These are prefixed with dangerous because, when used incorrectly, they can lead you into the same problems you'd have encountered with IntPtr. DangerousAddRef will increment the reference count and return a bool value to indicate success (true) or failure (false). DangerousRelease does the opposite — it decrements the reference count. If adding a reference fails — that is, the function returns false — you absolutely must not call the corresponding release function. This can lead to reference count imbalances and a dangling pointer!

DangerousGetHandle is just about the worst of all of these functions but nevertheless is sometimes necessary if you're calling a managed method that expects an IntPtr as an argument. It returns you the raw IntPtr that the SafeHandle is wrapping. Be very careful with this pointer; if the SafeHandle owns the handle (as is the case most of the time), it will not hesitate to close it while you're still using it. A general rule of thumb is that the SafeHandle must live at least as long as the inner handle you are working with. Otherwise, you'll end up with a dangling pointer once again. Remember, when you pass a SafeHandle to unmanaged code, it handles the process of fishing the underlying pointer out while at the same time ensuring the reference count remains correct. You needn't do it by hand.

A Simple SafeHandle Implementation

A concrete implementation of SafeHandle — for example, Microsoft.Win32.SafeHandles .SafeFileHandle — just supplies the acquisition and release routines for the handle. All of the other functions are inherited from SafeHandle. These new routines take form as a constructor (for acquisition) that sets the protected handle field and an override of ReleaseHandle (for release). As a brief example, consider a new SafeHandle to encapsulate a handle to a block of memory allocated with Marshal.AllocHGlobal:

 class HGlobalMemorySafeHandle : SafeHandleZeroOrMinusOneIsInvalid {     public HGlobalMemorySafeHandle(int bytes) : base(true)     {         SetHandle(Marshal.AllocHGlobal(bytes));     }     [ReliabilityContract(Consistency.WillNotCorruptState, Cer.Success)]     protected override bool ReleaseHandle()     {         Marshal.FreeHGlobal(handle);         return true;     } } 

This allocates a block of unmanaged memory in the constructor using the Marshal.AllocHGlobal method, and then frees that memory in the overridden SafeHandle.ReleaseHandle method. A client using this might do so as follows:

 using (SafeHandle sh = new HGlobalMemorySafeHandle(1024)) {     // Do something w/ ‘sh'...     // For example, marshal it across a P/Invoke boundary. } 

The client simply wraps the SafeHandle in a using statement that automatically invokes Dispose at the end. Assuming that the reference count is at zero (for example, you haven't exposed it to another thread that is in the process of using it), it will result in a call to the ReleaseHandle routine.

Consider if you then have a class that uses the SafeHandle internally. It is recommended that you hand out access to your handles. You shouldn't require your users to get at your SafeHandle in order to Dispose of it, for example. And as is the case with any other IDisposable type, if you own an instance and store it in a field, your type should likewise offer a Dispose method to get rid of the underlying resources:

 class MySafeHandleWrapper : IDisposable {     private SafeHandle memoryHandle;     public MySafeHandleWrapper()     {         memoryHandle = new HGlobalMemorySafeHandle(1024);     }     public void Dispose()     {         SafeHandle handle = memoryHandle;         if (handle != null && !handle.IsClosed)             handle.Dispose();     } } 

Notice that you don't have to worry about a finalizer. Should somebody forget to call Dispose on an instance of MySafeHandleWrapper (or fail to due to an asynchronous exception or rude shutdown getting in the way), then SafeHandle's finalizer will clean up the leaked resource at some nondeterministic point in the future. Even better than that, SafeHandle has a critical finalizer, which — as you will see shortly — executes in some circumstances where an ordinary finalizer would not. The bottom line is that this helps to prevent you from leaking resources on hosts such as SQL Server.

Notifying the GC of Resource Consumption

The GC is completely ignorant about what unmanaged resources your object holds on to. And thus, it cannot take such factors into account when calculating the overall system pressure. If it only knew your 8-byte object had allocated 10GB of unmanaged memory, it might have made it a higher priority to go ahead and collect it ASAP.

Memory Pressure

Memory pressure enables you to tell the GC about such things. Take our MyResourceManager type as an example. To the GC, an instance of this class simply occupies 4 + x bytes, where x is the general overhead for an object at runtime (implementation detail and subject to change). 4 represents the size of our IntPtr (assuming we're executing on a 32-bit platform; if we were on a 64-bit platform, the pointer would be 8 bytes). Unfortunately, this says nothing about the 1,024 additional bytes we allocated and which are pointed at by the ptr field.

The GC keeps a close eye on the state of memory in your program. It uses a heuristic to determine when it should initiate a collection and how far along in the generation list it should meander during a collection. If it knew that your object held on to such a large quantity of unmanaged memory, and that collecting it would relieve some of that memory footprint (which, in our case would happen due to our Finalize method), it might try to collect our 1K object sooner than it would have for a mere few bytes.

Thankfully, in version 2.0 of the .NET Framework, a new set of APIs has been introduced to solve this problem: GC.AddMemoryPressure and RemoveMemoryPressure. AddMemoryPressure instructs the GC that there is additional memory being used that it might not know about. RemoveMemoryPressure tells it when that memory has been reclaimed. Calls to the add and remove APIs should always be balanced. Otherwise, during long-running programs the pressure could get out of whack and cause collections to happen overly frequently or infrequently.

The pattern is to add pressure immediately once the memory has been allocated, and to reduce pressure immediately once the memory has been deallocated. In our example type MyResourceManager, this means the constructor and Dispose(bool) methods, respectively:

 sealed class MyResourceManager : IDisposable {     // Members not shown, e.g. fields, Dispose, Finalize, remain the same.     public MyResourceManager()     {         ptr = Marshal.AllocHGlobal(1024);         GC.AddMemoryPressure(1024);     }     private void Dispose(bool disposing)     {         if (ptr != IntPtr.Zero)         {             Marshal.FreeHGlobal(ptr);             ptr = IntPtr.Zero;             GC.RemoveMemoryPressure(1024);         }     } } 

This strategy is especially important for classes that don't offer a deterministic cleanup mechanism. Note that the efficiency of the memory pressure system works best when you add and remove in large quantities. The least amount you might consider is at the page level. One-megabyte quantities are even better.

Handle Collector

Many handles refer to scarce or limited system resources. Such resources might be managed by a semaphore (System.Threading.Semaphore) that tracks the total number of resources available, to avoid over-requesting an already exhausted resource. In this model, resource acquisitions require that clients decrement the semaphore first (which blocks if no resources are currently available); immediately after releasing the resource, a client must increment the semaphore to indicate availability (waking up other acquisition threads if appropriate). Such a strategy often works quite well. But if handles are not released deterministically, you could end up in a situation where other acquisition requests block until a handle gets released during finalization. This is nondeterministic and can lead to system unresponsiveness.

The HandleCollector type offers a way to force GC collections when the number of handles for a particular resource exceeds a specified threshold. This type lives in the System.Runtime.InteropServices namespace inside the System.dll assembly. You construct an instance, passing the threshold at which collections will occur, and then share that instance across your application for one resource handle type.

Upon every allocation and deallocation you must invoke Add and Remove, respectively. If adding a new reference would cause you to exceed the threshold a GC is triggered in hopes of clearing up any resources that were just waiting to be reclaimed during finalization. You should always call Add just before acquisition, and Remove just prior to release. The total number of handles allocated is available through the Count property.

Note that trying to Add doesn't block if the threshold is exceeded. This is merely a hint to the GC to attempt a collection. Protecting finite resources with a semaphore is still necessary.

Constrained Execution Regions

To understand why Constrained Execution Regions (CERs) are necessary, you first have to understand how hosts like SQL Server isolate and manage the code that they are running. In summary, such hosts tear down entire AppDomains to shut down managed code while still keeping the process (and other AppDomains within it) alive. Clearly in such situations any handle or memory leaks that live past an AppDomain shutdown can lead to unbounded growth over the long run. So the CLR invented new infrastructure in 2.0 to make releasing such resources more reliable.

CERs enable developers to write code that reliably cleans up resources. CERs come in the way of three features: CER blocks (initiated via a call to RuntimeHelpers.PrepareConstrainedRegions), critical finalizers (objects derived from CriticalFinalizerObject), and calls to RuntimeHelpers .ExecuteCodeWithGuaranteedCleanup. We'll see examples of each below. And CERs interact closely with the ReliabilityContractAttribute to ensure code being called is appropriate inside a CER.

A CER is eagerly prepared, meaning the CLR will allocate both heap and stack memory to avoid any out of memory or stack overflow conditions during its execution. To do this, the CLR must know a priori all of the code that will be called from inside of a CER. It then pre-JITs the code and ensures that enough stack space has been committed. While CERs are allowed to allocate memory explicitly, most have to be written under the assumption that doing so will trigger an out of memory condition. The Reliability ContractAttribute — which we will examine below — dictates the expectations the host has on a piece of code run inside a CER.

A detailed description would unfortunately take a chapter itself. Thankfully, most developers will never need to write a single CER in their life.

Reliable Cleanup in Aggressive Hosts

Some hosts monitor activity like memory allocations (in particular, those that would lead to paging to disk), stack overflows, and deadlocks. These are all situations that can prevent a program under the host's control from making forward progress. Worse, these situations can traditionally affect code throughout the entire process. Forward progress is necessary to guarantee that at least one piece of code is able to reach its goal, for example a stored procedure executing to completion. SQL Server would like to maintain a highly reliable, scalable, and performant execution environment for all code running on the server. In other words, it must maximize forward progress at all times. Thus, it does two things: (1) it isolates all logically related code inside an AppDomain, and (2) if it detects any problems in that code, it responds quickly by halting the rogue code.

Hosts may alter their policy through the CLR's hosting APIs. But, SQL, for example, responds to a situation like this by unloading the AppDomain. This means an asynchronous ThreadAbortException will be thrown in a targeted thread. There are some scenarios where that won't succeed: if the code on that thread is in a CER, catch or finally block, class constructor, or off in some unmanaged code function, for example, the host won't be able to actually raise the exception immediately. Instead, it will be raised as soon as the code leaves that block (assuming that it's not nested inside another). For more details on thread aborts, please refer to Chapter 10. If code doesn't respond in a timely manner, SQL will escalate to a rude shutdown. This process tears through all of the aforementioned protected blocks of code.

At that point, it's the responsibility of library code to ensure that there isn't process-wide (or worse, machine-wide) state corruption. But if a rude abort doesn't respect our right to execute finally blocks, for example, how can we guarantee that resources will be cleaned up? Furthermore, finalizers don't even get run during a rude abort, so we're surely going to leak some serious resources during a rude abort, aren't we? Well, actually no: the CLR does in fact execute critical finalizers during a rude abort. This is because critical finalizers rely on CERs to avoid a certain class of failures.

Critical Finalizers

Deriving from System.Runtime.ConstrainedExecution.CriticalFinalizerObject provides types the benefits of critical finalization. We already saw an example of such a class earlier in this chapter: SafeHandle. An implementation simply overrides the Finalize method (~TypeName() in C#), making sure to annotate it with [ReliabilityContract(Consistency.WillNotCorruptState, Cer.Success)], and doing its cleanup in there. Of course, it must abide by the constrained execution promises that this contract implies, defined further below.

For example:

 using System.Runtime.ConstrainedExecution; [ReliabilityContract(Consistency.WillNotCorruptState, Cer.Success)] class MyCriticalType : CriticalFinalizerObject {     private IntPtr handle;     MyCriticalType()     {        handle = OpenSomeHandle(...); // probably a p/invoke     }     ~MyCriticalType()     {         CloseHandle(handle);     } } 

It turns out very few classes in the Framework (other than SafeHandle) provide this guarantee, one notable type of which is SecureString. This is how SecureString makes good on its promise to prevent secured string data from staying in memory after an AppDomain shuts down.

Inline Critical Regions

You can introduce a CER block of code by calling RuntimeHelpers.PrepareConstrainedRegions just prior to entering a try block. This is a JIT intrinsic that causes the CLR to eagerly prepare your code. Only finally blocks are prepared in this case, not the try block itself. But all are prepared prior to entering the try block, so you are guaranteed that any cleanup code is able to execute:

 RuntimeHelpers.PrepareConstrainedRegions(); try {     // Unprotected, unconstrained code... } finally {     // Reliably prepared code... } 

Any code that must always execute uninterrupted goes into the finally block.

ExecuteCodeWithGuaranteedCleanup Critical Regions

The method RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup can be used as a simple way to initiate a critical region of code. It accepts two delegate parameters, TryCode code and CleanupCode backoutCode, and an object which gets passed to those delegates. The delegate signatures are of the form:

 public delegate void TryCode(object userData); public delegate void CleanupCode(object userData, bool exceptionThrown); 

The semantics of this function are identical to the manually prepared try block above. In other words, a call to ExecuteCodeWithGuaranteedCleanup is conceptually equivalent to:

 void ExecuteCodeWithGuaranteedCleanup(TryCode code,     CleanupCode backoutCode, object userData) {     bool thrown = false;     RuntimeHelpers.PrepareConstrainedRegions();     try     {         code(userData);     }     catch     {         thrown = true;         throw;     }     finally     {         backoutCode(userData, thrown);     } } 

The backoutCode delegate is executed inside a CER and, thus, should abide by the constraints imposed on critical regions. The code delegate executed inside the try block is not.

Reliability Contracts

Code inside constrained regions make a set of implicit agreements with the host. The host returns the favor by permitting the code to execute reliably to completion. Some of the promises are difficult to fulfill; in fact, most people refer to writing correct CER code as being rocket science. One of the major difficulties is that there is no systematic way to prove that you've gotten it right (other than aggressive testing). There is no tool, for example, that will detect violations of a contract.

Methods annotated with the ReliabilityContractAttribute declare what guarantees the code makes. There are two parts to this guarantee: consistency and success. These indicate to the host whether the code can fail, and if so, what the risk of corrupt state is. These are represented by two arguments to the attribute, of types Consistency and Cer:

• Consistency is an enumeration that offers four values: MayCorruptInstance, MayCorrupt AppDomain, MayCorruptProcess, and WillNotCorruptState. The first three indicate that if the method doesn't execute to completion, it might have corrupted state. In order, they say that instance-level corruption (a single object), AppDomain corruption (cross-thread), or process-level corruption could occur. Each is of increasing severity and dictates how aggressive the host responds upon failure. WillNotCorruptState is hard to achieve in practice. It guarantees that no state can be corrupted. Notice that critical finalizers must execute under this guarantee.

• Cer has two legal values: MayFail tells the host that it can expect a possible failure during execution. If a failure occurs, the host can inspect Consistency to determine the degree of corruption that might have occurred. Success indicates that a given CER method will never fail when executed. This, like WillNotCorruptState, is difficult to achieve in practice. Notice that critical finalizers also execute under this constraint.

So reliable code must first and foremost declare its reliability and failure guarantees. But furthermore, such code may only call other methods with at least the same level of guarantee. And perhaps worst of all, in cases such as [ReliabilityContract(Consistency.WillNotCorruptState, Cer.Success)], you cannot even allocate any memory! This means voiding any usage of the newobj and box IL instructions, for example, among other things. Since most Framework APIs aren't annotated with reliability contracts, this means most of it is off limits inside a CER. This can make writing code very difficult.

CERs execute as though they are running with a contract of:

 [ReliabilityContract(Consistency.MayCorruptInstance, Cer.MayFail)] 

And furthermore, CERs may only call other methods with these reliability contracts:

 [ReliabilityContract(Consistency.WillNotCorruptState, Cer.MayFail)] [ReliabilityContract(Consistency.WillNotCorruptState, Cer.Success)] [ReliabilityContract(Consistency.MayCorruptInstance, Cer.MayFail)] [ReliabilityContract(Consistency.MayCorruptInstance, Cer.Success)] 

Notice that these are all either equal to or more stringent reliability requirements, consistent with the explanation above.

If you attempt to violate your reliability contract, you can cause damaging memory leaks or corruption: if the host notices, you can be assured it will try to prevent you from doing so. But in some cases, it won't detect such things. If you promise that you will only corrupt instance state, yet go and corrupt process-wide state, the host will not notice. Instead, it will blindly trudge forward, potentially causing even more damage to the state of your application or data.