Integer Overflows

Integer Overflows

Integer overflows are one of my favorite problems. I gained a healthy respect for the limits of how a computer represents data while writing code to simulate airfoils. Large matrix manipulation using floating-point arithmetic will give you a number of lessons in the school of hard knocks. Most programmers deal only with integer types, and there are only two major classes of problems you might encounter. Let's take a look at signed-unsigned mismatches. Consider the following code:

int Example(char* str, int size) { char buf[80]; if(size < sizeof(buf)) { //Should be safe strcpy(buf, str); } }

Quick, what's the problem here? If you didn't spot it immediately, here it is: any native integer type is almost always signed. But sizeof returns a size_t type, which is unsigned. What if the caller managed to pass in negative size? Assuming that the compiler casts sizeof(buf) to a signed integer for you, the comparison will succeed and you'll overflow your buffer. The solution is to always declare your integers as unsigned unless you explicitly require negative numbers. Most systems will treat an integer that isn't explicitly declared as unsigned as signed. Fortunately, the compiler will report signed-unsigned mismatches unless the programmer has gone in and cast away the warnings. Examine string length comparisons very closely, and don't ignore signed-unsigned mismatch warnings without careful examination. If the programmer has cast away warnings, examine these carefully a security bug could be lurking!

Here's another way to cause problems: adding one to MAX_INT. If you have code that adds some predetermined amount of storage for a trailing delimiter, make sure to do your size checking before you add to it, or alternately, explicitly check for the overflow with this:

if(result < original) { //Error! return false; }

This is actually a common problem when using GetTickCount to determine how long something has run. GetTickCount rolls over about every 40 days, so if you're using this, make sure and catch this condition.

Integer overruns are another area where you can create some interesting bugs. Consider the following data type:

typedef struct _LSA_UNICODE_STRING { USHORT Length; USHORT MaximumLength; PWSTR Buffer; } LSA_UNICODE_STRING;

In this case, the Length and MaximumLength members store the number of bytes that the buffer can contain, which would allow for 32,768 Unicode characters. Let's look at a possible implementation of a function that takes in a WCHAR pointer and initializes one of these structures:

void InitLsaUnicodeString(const WCHAR* str, LSA_UNICODE_STR* pUnicodeStr) { if(str == NULL) { pUnicodeStr->Buffer = NULL; pUnicodeStr->Length = 0; pUnicodeStr->MaximumLength = 0; } else { unsigned short len = (unsigned short)wcslen(str) * sizeof(WCHAR); pUnicodeStr->Buffer = str; pUnicodeStr->Length = len; pUnicodeStr->MaximumLength = len; } }

Examine the code carefully; consider what happens if someone passes in a string that is 32,769 bytes. If a computer is nearby, pop up an instance of calc.exe and follow along. Let's multiply that by 2. Now switch to hexadecimal display, and you'll see that the length is 0x10002. Once we cast the result to an unsigned short, we now see that the Length field has just been set to 2! Now to complete the train wreck, imagine this LSA_UNICODE_STRING structure gets passed to another function that merely checks whether Length is less than the MaximumLength of the destination and calls wcscpy! Be extremely careful when truncating integers here's how the code could be improved:

 unsigned long len = wcslen(str) * sizeof(WCHAR); if(len > 0xffff) { pUnicodeStr->Buffer = NULL; pUnicodeStr->Length = 0; pUnicodeStr->MaximumLength = 0; }

Now let's look at another way that we can bungle integers; integer multiplication can get a little tricky. Take a look at this example:

int AllocateStructs(void** ppMem, unsigned short StructSize, unsigned short Count) { unsigned short bytes_req; bytes_req = StructSize * Count; *ppMem = malloc(bytes_req); if(*ppMem == NULL) return -1; else return 0; }

As in the LSA_UNICODE_STRING example, it's possible that the multiplication could result in an overflow, which would lead to our allocating a buffer that's much too small for the job and the subsequent copy into the buffer would cause an overflow. In this example, declaring bytes_req as an unsigned integer would overcome the problem. Here's a more robust way to deal with the general problem:

int AllocateStructs(void** ppMem, unsigned short StructSize, unsigned short Count) { unsigned short bytes_req; if(StructSize == 0 Count > 0xffff/StructSize) { assert(false); return -1; } bytes_req = StructSize * Count; *ppMem = malloc(bytes_req); if(*ppMem == NULL) return -1; else return 0; }

If a program has custom memory allocation routines, it's a fairly common error to not account for integer overflows, and a straightforward example like this one could be hidden within complicated code that makes sure you allocate only blocks of a certain size. Any time you see a multiplication operation conducted on an integer, ask yourself what happens if this causes the integer to wrap around.

Another interesting aspect of integer overflows is the fact that a pointer is just an unsigned integer containing a memory location. Pointer arithmetic is prone to exactly the same types of problems as we've outlined with other types of integer math. Any time someone is doing pointer arithmetic, check to be sure that there aren't integer overflows. One thing to remember is that this is an area where the attackers are currently looking for problems. Simple string-based buffer overflows are getting more and more difficult to find in production code, and so the attackers are starting to look for more subtle types of errors.

A Related Issue: Integer Underflows

Imagine you have code like this:

void AllocMemory(size_t cbAllocSize) { //We don't accommodate for trailing '\0' cbAllocSize--; char *szData = malloc(cbAllocSize); ... }

On the surface, it looks fine, until you realize that bad things can happen if cbAllocSize == 0! Bad things could happen on two fronts. If the code does not check that szData != NULL, or if cbAllocSize wraps to 1, you have a problem! In the case of cbAllocSize (a signed integer), -1 becomes 4 billion or so on a high-end server that has over 4 GB of RAM. The moral of this story is be wary of code that could potentially underflow less than zero.



Writing Secure Code
Writing Secure Code, Second Edition
ISBN: 0735617228
EAN: 2147483647
Year: 2001
Pages: 286

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net