Knowing When to Use the Win32 API

There’s a good reason why the Win32 API topics appear at the end of this book—you should view them as the option of last resort. It’s not that the Win32 API is inherently difficult to use (although that’s one reason to avoid it), but the fact that you have considerable resources you can rely on in the .NET Framework. You should exhaust all of the other options before you attempt to use the Win32 API to solve a security problem. However, some situations do call for the Win32 API. Here’s a list of the common tasks you’ll perform.

• Interoperability with older Win32 applications

• Working with Active Directory

• Managing some types of COM+ applications

• Accessing messages in Microsoft Message Queuing Services (MSMQ)

• Filling a security gap not supported by the .NET Framework

  Tip

  One of the issues you must consider as a developer is when the Win32 API is an appropriate alternative to using .NET Framework security features. However, many developers don’t consider the third solution—hardening the server to make it less likely that a cracker will gain entry without significant effort. According to an InfoWorld article (http://www.infoworld.com/article/03/04/29/HNmsguide_1.html), Microsoft is creating guides that demonstrate how to harden a server. You can find the Windows 2003 Server version of the guide at http://www.microsoft.com/technet/security/prodtech/windows/win2003/w2003hg/sgch00.asp. The Windows 2000 Server version of the guide appears at http://www.microsoft.com/technet/security/prodtech/windows/win2khg/default.asp.

Win32 API and .NET Framework Differences

The Win32 API has several advantages in its favor that the .NET Framework simply can’t duplicate. For example, the Win32 API has an advantage in speed—security checks are nearly instantaneous. Even on a fast machine, some .NET Framework checks can incur a noticeable delay in seconds. When you ask a technology to do more, it requires more resources to answer the request.

It’s also important consider the complexity and the simplicity of the Win32 API. From a coding perspective, the Win32 API requires a lot of code and most of it’s hard to understand.

For the most part, the .NET Framework makes programming considerably easier once you know what you want to do and how you want to do it. However, the Win32 API has a simpler security model. All you really need to understand is that every caller has a SID. This SID appears on a resource’s access list or it doesn’t. When a SID doesn’t appear on the access list, the caller doesn’t have the right to use the resource. The .NET Framework model means you not only have to track both code access and role-based security requirements, but you also need to decide which method to use for implementing that security, imperative or declarative. The Win32 API is simultaneously easier and harder to use than the .NET Framework.

Avoiding Dangerous APIs

One of the security issues with the Win32 API is that the code is unmanaged. When an application needs memory, it requests the memory from Windows and then returns the memory to the pool when finished—at least if everything works as anticipated. The Win32 API doesn’t perform any type checking, so it’s possible to make dangerous data coercions and create an environment where buffer overruns are difficult to avoid. Fortunately, Windows does perform some error checking and CLR performs some management checks when working with unmanaged code, so you don’t have to worry that every function is a security time bomb ready to explode in your face.

The sections that follow describe features that make an API dangerous from a security perspective. You should treat functions and function classes that fall into these categories with special care. It’s still possible to use them safely, but you have to be aware of the limitations these functions have when working with them and include additional security checks to avoid possible problems.

Using Old Functions

Microsoft has expanded the Win32 API over the years to meet specific needs, to adapt to changing conditions, and to fix errors. In most cases, Microsoft gave the updated function a new, but similar, name. In many cases, Microsoft simply added a number or the word Ex to the function name. For example, you might see MyFunction(), MyFunctionEx(), and MyFunction3() listed in the Win32 API. Whenever a new function offers significantly different functionality, Microsoft attempts to keep the old function in place and simply gives the new function the expanded name.

Because Microsoft doesn’t document why it added the new function, in most cases, you need to consider the old versions as less safe from a security perspective than the newer function. The new function could add functionality that you need to maintain a secure environment—the older version of the function might work fine in the desktop environment, but could cause problems on a LAN. In some cases, developers became so used to the broken operation of a function that Microsoft retained the broken function and added a new one that fixed the problem. Obviously, you don’t want to add a broken function to your code.

Note

Some developers might argue that Microsoft has tested the older function better and therefore the older function contains fewer security holes. This argument doesn’t hold up well in light of Microsoft’s policy of not changing broken functions for fear of also damaging code that relies on the faulty function.

Using Unsafe Code

Only C# developers need to worry about unsafe code—at least in the way this section uses the term. The best definition for unsafe code is unmanaged code that appears within a managed environment. Any code that relies on the use of manual pointers (* symbol) or addresses (& symbol) is unsafe code. Whenever you write code that uses these symbols, you also need to use the unsafe keyword in the method declaration as shown in Listing 14.1. (You can find this example in the \Chapter 14\C#\Unsafe folder of the source code located on the Sybex Web site.)

Listing 14.1 Using Unsafe Code in a C# Program

unsafe private void btnTest_Click(object sender, System.EventArgs e) {    int   Input = Int32.Parse(txtInput.Text); // Input string.    // Convert the input value.    DoTimeIt(&Input);    // Display the result    txtOutput.Text = Input.ToString(); } unsafe private void DoTimeIt(int* Input) {    int   Output;  // Output to the caller.    // Display the current minute.    txtMinute.Text = System.DateTime.Now.Minute.ToString();    // Create the output value.    Output = *Input;    Output = Output * System.DateTime.Now.Minute;    // Output the result.    *Input = Output; }

This is a simple example that you create using other methods, but it demonstrates a principle you’ll need to create applications that rely on the Win32 API. The btnTest_Click() accesses the input value, converts it to an int, and supplies the address of the int to the DoTimeIt() method. Because the code relies on an address rather than the value, any change in the supplied value by DoTimeIt() will remain when the call returns.

The DoTimeIt() method accesses the current time, multiplies it by the value of the input string, and then outputs the value. Notice the use of pointers in this method to access the values contained in the Input and Output variables. The reason this code is unsafe is that the compiler can’t check it for errors. For example, you could replace the last line with Input = &Output; and the compiler would never complain, but you also wouldn’t see the results of the multiplication.

From a security perspective, the inability to check for errors is a disaster in the making. Whenever you lose the ability to verify the integrity of the data you’re working with, you give crackers an opportunity to change that data. Even if you later verify that the data is of the right type and correct length, and contains the proper information, you have no way of knowing whether an external force compromised the data or the surrounding memory when you weren’t able to view the pointer.

Generally, you should avoid using unsafe code whenever possible—if for no other reason than to get as much help as possible from the compiler. The “Using Pointer Alternatives” section tells you about managed alternatives that mimic pointers. In short, while unsafe code is a necessity when working with the Win32 API, you should avoid it whenever possible.

Using Pointer Alternatives

One of the first issues that you’ll face when working with the Win32 API is the use of pointers—the Win32 API uses them by the gross. The Win32 API uses pointers as function arguments: they appear within data structures, and you even find them nested within each other. The problem with pointers is that they aren’t objects—they really aren’t anything. A pointer is an abstraction, an address for something real. As a real world example, the pointer to your house is the street address found on letters and packages. The .NET Framework refrains from relying on pointers (from a developer’s perspective) and uses the actual object whenever possible. The pointers are still there; CLR simply manages them for you. You can use this fact to your advantage in avoiding pointers by making use of pointer alternatives.

The first pointer alternative to consider is that pointers aren’t always necessary. Avoid pointers, whenever possible, by verifying the need to use them. In many situations, you can simply pass a value to the Win32 API when your application doesn’t require a return value. CLR still converts the value to a pointer for you—the point is that the conversion occurs in a managed environment and all of the details are taken care of for you. Passing a value is safe from a security standpoint because your code won’t receive any contamination on return from the call.

A second pointer alternative to consider is the use of the out or the ref keywords in C#. When you use these keywords, CLR automatically creates a pointer to the object, data structure, or value for you. (When using the out keyword, you let the function you call initialize the memory.) Visual Basic supports the ref keyword using the ByRef keyword. However, it doesn’t have a direct substitute for the out keyword, which can make things a little difficult. In many cases, you can substitute the [OutAttribute] attribute as part of the function declaration, but this feature doesn’t always work and you might find that you get an error return from the Win32 API function. Of the two options, using the ref keyword is the most secure because your code controls the memory. However, the ref keyword is also the most work because your code is also responsible for allocating and freeing the memory from the global heap.

Using an IntPtr (essentially a managed pointer) is a third option. An IntPtr easily handles a variety of tasks, including a pointer to a pointer (such as void** used in Visual C++). In fact, you can place this use of an IntPtr in your rules of thumb book. Generally, you can replace any pointer to a pointer (such as a handle) with an IntPtr for all Win32 API calls. The IntPtr is problematic from a security perspective because you have less control over it than you do using any other technique discussed in this section.

Sometimes you must use a pointer—there simply isn’t any way around the issue. For example, you’ll often find that COM calls require pointers to pointers, such as when you want to work with an interface. In this situation, you might find it impossible to develop a substitute for pointers. When this problem occurs, try to localize the pointer code to a special function, even if it might not make sense to create a separate function from a program flow perspective. Placing the pointer in its own function makes it easier to work with the pointer, reduces the probability of missed pointer errors, and makes it easier to debug the application later. Most importantly, by isolating the problem code, you increase the chance that you’ll catch security issues before they become a problem.