The Sin Explained | Writing Secure Code

The effects of integer errors range from crashes and logic errors to escalation of privilege and execution of arbitrary code. A current incarnation of the attack hinges on causing an application to make errors in allocating code; the attacker is then able to exploit a heap overflow. If you typically develop in a language other than C/C++, you may think youre immune to integer overflows, but this would be a mistake. Logic errors related to the truncation of integers have resulted in a bug in Network File System (NFS) where any user can access files as root.

Sinful C and C++

Even if youre not a C or C++ programmer, its worthwhile to look at the dirty tricks that C/C++ can play on you. Being a relatively low-level language, C sacrifices safety for execution speed, and has the full range of integer tricks up its sleeve. Most other languages wont be able to do all of the same things to your application, and some, like C#, can do unsafe things if you tell them to. If you understand what C/C++ can do with integers, youll have a better shot at knowing when youre about to do something wrong, or even why that Visual Basic .NET application keeps throwing those pesky exceptions. Even if you only program in a high-level language, youll eventually need to make system calls, or access external objects written in C or C++. The errors you made in your code can show up as overflows in the code you call.

Casting Operations

There are a few programming patterns and issues that most frequently lead to integer overflows. One of the first is a lack of awareness of casting order and implicit casts from operators. For example, consider this code snippet:

 const long MAX_LEN = 0x7fff; short len = strlen(input); if(len < MAX_LEN)  //do something

Aside from truncation errors, whats the order of the cast that happens when len and MAX_LEN are compared? The language standard states that you have to promote to like types before a comparison can occur; so what youre really doing is up-casting len from a signed 16-bit integer to a signed 32-bit integer. This is a straightforward cast because both types are signed. In order to maintain the value of the number, the type value is sign extended until it is the same size as the larger type. In this case, you might have this as a result:

 len = 0x0100; (long)len = 0x00000100;

 len = 0xffff; (long)len = 0xffffffff;

As a result, if the attacker can cause the value of len to exceed 32K, len becomes negative, because once its up-cast to a 32-bit long its still negative; and your sanity check to see if len is larger than MAX_LEN sends you down the wrong code path .

Here are the conversion rules for C and C++:

Signed int to Larger signed int

The smaller value is sign-extended; for example, (char)0x7f cast to an int becomes 0x0000007f, but (char)0x80 becomes 0xffffff80.

Signed int to Same-Size unsigned int

The bit pattern is preserved, though the value may or may not change. So (char)0xff (-1) remains 0xff when cast to an unsigned char, but -1 clearly has a different meaning than 255.

Signed int to Larger unsigned int

This combines the two behaviors: The value is first sign-extended to a signed integer, and then cast to preserve the bit pattern. This means that positive numbers behave as youd expect, but negative numbers might yield unexpected results. For example, (char)-1 (0xff) becomes 4,294,967,295 (0xffffffff) when cast to an unsigned long.

Unsigned int to Larger unsigned int

This is the best case: the new number is zero-extended, which is generally what you expect. Thus (unsigned char)0xff becomes 0x000000ff when cast to an unsigned long.

Unsigned int to Same-Size signed int

As with the cast from signed to unsigned, the bit pattern is preserved, and the meaning of the value may change, depending on whether the uppermost (sign) bit is a 1 or 0.

Unsigned int to Larger signed int

This behaves very much the same as casting from an unsigned int to a larger unsigned int. The value first zero-extends to an unsigned int the same size as the larger value, then is cast to the signed type. The value of the number is maintained , and wont usually cause programmer astonishment.

Downcast

Assuming that any of the upper bits are set in the original number, you now have a truncation, which can result in general mayhem. Unsigned values can become negative or data loss can occur. Unless youre working with bitmasks , always check for truncation.

Operator Conversions

Most programmers arent aware that just invoking an operator changes the type of the result. Usually, the change will have little effect on the end result, but the corner cases may surprise you. Heres some C++ code that explores the problem:

 template <typename T> void WhatIsIt(T value) {  if((T)-1 < 0)  printf("Signed");  else  printf("Unsigned");  printf(" - %d bits\n", sizeof(T)*8); }

For simplicity, well leave out the case of mixed floating point and integer operations. Here are the rules:

If either operand is an unsigned long, both are upcast to an unsigned long. Academically, longs and ints are two different types, but on a modern compiler, theyre both 32-bit values; and for brevity, well treat them as equivalent.
In all other cases where both operands are 32-bits or less, the arguments are both upcast to int, and the result is an int.

Most of the time, this results in the right thing happening, and implicit operator casting can actually avoid some integer overflows. There are some unexpected consequences, however. The first is that on systems where 64-bit integers are a valid type, you might expect that because an unsigned short and a signed short get upcast to an int, and the correctness of the result is preserved because of the operator cast (at least unless you downcast the result back to 16 bits), an unsigned int and a signed int might get cast up to a 64-bit int (_int64). If you think it works that way, youre unfortunately wrongat least until the C/C++ standard gets changed to treat 64-bit integers consistently.

The second unexpected consequence is that the behavior also varies depending on the operator. The arithmetic operators (+, - , *, /, and %) all obey the preceding rules as youd expect. What you may not expect is that the binary operators (&, , ^) also obey the same rules; so, (unsigned short) (unsigned short) yields an int! The Boolean operators (&&, , and !) obey the preceding rules in C programs, but return the native type bool in C++. To further add to your confusion, some of the unary operators tamper with the type, but others do not. The ones complement (~) operator changes the type of the result; so ~((unsigned short)0) yields an int, but the pre and postfix increment and decrement operators (++, -- ) do not change the type.

As an illustration, a senior developer with many years of experience proposed using the following code to check whether two unsigned 16-bit numbers would overflow when added together:

 bool IsValidAddition(unsigned short x, unsigned short y) {  if(x + y < x)  return false;  return true; }

It looks like it ought to work. If you add two positive numbers together and the result is smaller than either of the inputs, you certainly have a malfunction. The exact same code does work if the numbers are unsigned longs. Unfortunately for our senior developer, it will never work because the compiler will optimize out the entire function to true!

Recalling the preceding behavior, whats the type of unsigned short + unsigned short? Its an int. No matter what we put into two unsigned shorts, the result can never overflow an int, and the addition is always valid. Next , you need to compare an int with an unsigned short. The value x is then cast to an int, which is never larger than x + y. To correct the code, all you need to do is cast the result back to an unsigned short, like so:

 if((unsigned short)(x + y) < x)

The same code was shown to a blackhat who specializes in finding integer overflows, and he missed the problem as well, so the experienced developer has plenty of company!

Arithmetic Operations

Be sure and understand the implications of casts and operator casts when thinking about whether a line of code is correctan overflow condition could depend on implicit casts. In general, you have four major cases to consider: unsigned and signed operations involving the same types, and mixed-type operations that could also be mixed sign. The simplest of all is unsigned operations of the same type; signed operations have more complexity; and when youre dealing with mixed types, you have to consider casting behavior. Well cover example defects and remedies for each type of operation in later sections.

Addition and Subtraction

The obvious problem with these two operators is wrapping around the top and bottom of the size you declared. For example, if youre dealing with unsigned 8-bit integers, 255 + 1 = 0. Or 2 3 = 255. In the signed 8-bit case, 127 + 1 = 128. A less obvious problem happens when you use signed numbers to represent sizes. Now someone feeds you a size of 20, you add that to 50, come up with 30, allocate 30 bytes, and then proceed to copy 50 bytes into the buffer. Youre now hacked. Something to remember, especially when dealing with languages where integer overflows are anywhere from difficult to impossible , is that subtracting from a positive and getting less than you started with is a valid operation; it wont throw an overflow exception, but you may not have the program flow you expect. Unless youve previously range checked your inputs and are certain that the operation wont overflow, be sure to validate every operation.

Multiplication, Division, and Modulus

Unsigned multiplication is fairly straightforward: any operation where a * b > MAX_INT results in an incorrect answer. A correct, but less efficient way to check the operation, is to convert your test to b > MAX_INT/a. A more efficient way to check the operation is to store the result in the next larger integer where available, and then see if there was an overflow. For small integers, the compiler will do that for you. Remember that short * short yields an int. Signed multiplication requires one extra check to see if the answer wrapped in the negative range.

You may be wondering how division, other than dividing by zero, can be a problem. Consider a signed 8-bit integer: MIN_INT = 128. Now divide that by 1. Thats the same thing as writing (128). The negation operator can be rewritten as ~x + 1. The ones complement of 128 (0x80) is 127, or 0x7f. Now add 1, and you get 0x80! So you see that minus negative 128 is still minus 128! The same is true of any minimum signed integer divided by 1. If youre not convinced that unsigned numbers are easier to validate yet, we hope this convinces you.

The modulus (remainder) operator returns the remainder of a division operation; thus, the answer can never have a larger magnitude than the numerator. You may be wondering how this can overflow. It cant actually overflow, but it can return an incorrect answer, and this is due to casting behavior. Consider an unsigned 32-bit integer that is equal to MAX_INT, or 0xffffffff, and a signed 8-bit integer that has a value of 1. So 1 mod 4,294,967,295 ought to yield 1, right? Not so fast. The compiler wants to operate on like numbers, so the 1 has to be cast to an unsigned int. Recall from earlier how that happens. First you sign-extend until you get to 32 bits, so youll convert 0xff to 0xffffffff. It then converts (int)(0xfffffff) to (unsigned int)(0xffffffff). You see that the remainder of 1 divided by 4 billion is zero, or at least according to our computer! The same problem will occur any time youre dealing with unsigned 32- or 64-bit integers mixed with negative signed integers, and it applies to division as well1/4,294,967,295 is really 1, which is annoying when youve expected to get zero.

Comparison Operations

Surely something as basic as equality ought to work, or so one would hope. Unfortunately, if youre dealing with mixed signed and unsigned integers, theres no such guaranteeat least if the signed value isnt a larger type than the unsigned value. The same problem we outlined with division and modulus will cause problems.

Another way that comparison operations will get you is when you check for a maximum size using a signed value: your attacker finds some way to cause the value to be negative, and thats always less than the upper limit you expected. Either use unsigned numbers, which is what we recommend, or be prepared to make two checks: First that the number is greater than or equal to zero, and second that it is smaller than your limit.

Binary Operations

Binary operations, like binary AND, OR, and XOR (exclusive or), ought to work, but again, sign extension will mix things up. Lets look at an example:

 int flags = 0x7f; char LowByte = 0x80; if((char)flags ^ LowByte == 0xff)  return ItWorked;

You might think that the result of this operation ought to be 0xff, which is what youre checking for, but then the pesky compiler gets ambitious and casts both values to an int. Recall from our operator conversions that even binary operations convert to int when given smaller valuesso flags gets extended to 0x0000007f, which is just fine, but LowByte gets extended to 0xffffff80, and our result is really 0xfffffffff!

Sinful C#

C# is very much like C++, which makes it a nice language if you already understand C/C++, but in this case, C# has most of the same problems C++ has. One interesting aspect of C# is that it enforces type safety much more stringently than C/C++ does. For example, the following code throws an error:

 byte a, b; a = 255; b = 1; byte c = (b + a); error CS0029: Cannot implicitly convert type 'int' to 'byte'

If you understand what this error is really telling you, youll think about the possible consequences when you get rid of the error by writing:

 byte c = (byte)(b + a);

A safer way to get rid of the warning is to invoke the Convert class:

 byte d = Convert.ToByte(a + b);

If you understand what the compiler is trying to tell you with all these warnings, you can at least think about whether theres really a problem. However, there are limits to what it can help with. In the preceding example, if you got rid of the warning by making a, b, and c signed ints, then overflows are possible, and youd get no warning.

Another nice feature of C# is that it uses 64-bit integers when it needs to. For example, the following code returns an incorrect result when compiled in C, but works properly on C#:

 int i = -1; uint j = 0xffffffff; //largest positive 32-bit int if(i == j)  Console.WriteLine("Doh!");

The reason for this is that C# will upcast both numbers to a long (64-bit signed int), which can accurately hold both numbers. If you press the issue and try the same thing with a long and a ulong (which are both 64-bit in C#), you get a compiler warning that you need to convert one of them explicitly to the same type as the other. Its the authors opinion that the C/C++ standard should be updated so that if a compiler supports 64-bit operations, it should behave as C# does in this respect.

Checked and Unchecked

C# also supports the checked and unchecked keywords. You can declare a block of code as checked as this example shows:

 byte a = 1; byte b = 255; checked {  byte c = (byte)(b + a);  byte d = Convert.ToByte(a + b);  Console.Write("{0} {1}\n", b+1, c); }

In this example, the cast of a + b from int to byte throws an exception. The next line, which calls Convert.ToByte(), would have thrown an exception even without the checked keyword, and the addition within the arguments to Console.Write() throws an exception because of the checked keyword. Because there are times where integer overflows are intentional, the unchecked keyword can be used to declare blocks of code where integer overflow checking is disabled.

You can also use both checked and unchecked to test individual expressions as follows :

 checked(c = (byte)(b + a));

A third way to enable checked behavior is through a compiler optionpassing in /checked to the compiler on the command line. If the checked compiler option is enabled, youll need to explicitly declare unchecked sections or statements where integer overflows are actually intended.

Sinful Visual Basic and Visual Basic .NET

Visual Basic seems to undergo periodic language revisions, and the transition from Visual Basic 6.0 to Visual Basic .NET is the most significant revision since the shift to object-oriented code in Visual Basic 3.0. One of the more fundamental changes is in the integer types, as shown in Table 3-1.

Table 3-1: Integer Types Supported by Visual Basic 6.0 and visual Basic .NET
Integer Type	Visual Basic 6.0	Visual Basic .NET
Signed 8-bit	Not supported	System.SByte
Unsigned 8-bit	Byte	Byte
Signed 16-bit	Integer	Short
Unsigned 16-bit	Not supported	System.UInt16
Signed 32-bit	Long	Integer
Unsigned 32-bit	Not supported	System.UInt32
Signed 64-bit	Not supported	Long
Unsigned 64-bit	Not supported	System.UInt64

In general, both Visual Basic 6.0 and Visual Basic .NET are immune to execution of arbitrary code through integer overflows. Visual Basic 6.0 throws run-time exceptions when overflows happen in either an operator or in one of the conversion functionsfor example, CInt(). Visual Basic .NET throws an exception of type System.OverflowException. As detailed in Table 3-1, Visual Basic .NET also has access to the full range of integer types defined in the .NET Framework.

Although operations within Visual Basic itself may not be vulnerable to integer overflows, one area that can cause problems is that the core Win32 API calls all typically take unsigned 32-bit integers (DWORD) as parameters. If your code passes signed 32-bit integers into system calls, its possible for negative numbers to come back out. Likewise, it may be completely legal to do an operation like 2 8046 with signed numbers, but with an unsigned number, that represents an overflow. If you get into a situation where youre obtaining numbers from a Win32 API call, manipulating those numbers with values obtained from or derived from user input, and then making more Win32 calls, you could find yourself in an exploitable situation. Switching back and forth between signed and unsigned numbers is perilous. Even if an integer overflow doesnt result in arbitrary code execution, unhandled exceptions do cause denial of service. An application that isnt running isnt making any money for your customer.

Sinful Java

Unlike Visual Basic or C#, Java has no defense against integer overflows. As documented in the Java Language Specification , found at http://java.sun.com/docs/books/jls/second_edition/html/typesValues.doc.html#9151 :

The built-in integer operators do not indicate overflow or underflow in any way. The only numeric operators that can throw an exception (11) are the integer divide operator / (15.17.2) and the integer remainder operator % (15.17.3), which throw an ArithmeticException if the right-hand operand is zero.

Like Visual Basic, Java also only supports a subset of the full range of integer types. Although 64-bit integers are supported, the only unsigned type is a char, which is a 16-bit unsigned value.

Due to the fact Java only supports signed types, most of the overflow checks become tricky; and the only area where you dont run into the same problems as C/C++ is when mixing signed and unsigned numbers would lead to unexpected results.

Sinful Perl

Although at least two of the authors of this book are enthusiastic supporters of Perl, Perls integer handling is best described as peculiar. The underlying type is a double-precision floating point number, but testing reveals some interesting oddities . Consider the following code:

 $h = 4294967295; $i = 0xffffffff; $k = 0x80000000; print "$h = 4294967295 - $h + 1 = ".($h + 1)."\n"; print "$i = 0xffffffff - $i + 1 = ".($i + 1)."\n"; printf("\nUsing printf and %%d specifier\n"); printf("$i = %d, $i + 1 = %d\n\n", $i, $i + 1); printf("Testing division corner case\n"); printf("0x80000000/-1 = %d\n", $k/-1); print "0x80000000/-1 = ".($k/-1)."\n";

The test code yields the following results:

 [e:\projects_sins]perl foo.pl 4294967295 = 4294967295 - 4294967295 + 1 = 4294967296 4294967295 = 0xffffffff - 4294967295 + 1 = 4294967296 Using printf and %d specifier $i = -1, $i + 1 = -1 Testing division corner case 0x80000000/-1 = -2147483648 0x80000000/-1 = -2147483648

At first, the results look peculiar, especially when using printf with format strings, as opposed to a regular print statement. The first thing to notice is that youre able to set a variable to the maximum value for an unsigned integer, but adding 1 to it either increments it by 1, or, if you look at it with %d, does nothing. The issue here is that youre really dealing with floating point numbers, and the %d specifier causes Perl to cast the number from double to int. Theres not really an internal overflow, but it does appear that way if you try to print the results.

Due to Perls interesting numeric type handling, we recommend being very careful with any Perl applications where significant math operations are involved. Unless you have prior experience with floating point issues, you could be in for some interesting learning experiences. Other higher-level languages, such as Visual Basic, will also sometimes internally convert upwards to floating point as well. The following code and result shows you exactly whats going on:

 print (5/4)."\n";1.25

For most normal applications, Perl will just do the right thing, which it is exceedingly good at. However, dont be fooled into thinking that youre dealing with integersyoure dealing with floating point numbers, which are another can of worms entirely.