function of Program 1.1.

1.6 Buffer Overflows for Breaking and Entering

This section presents a simplified explanation of buffer overflows and how they might be used to attack a computer system. A buffer overflow occurs when a program copies data into a variable for which it has not allocated enough space.

Example 1.4 shows a code segment that may have a buffer overflow. A user types a name in response to the prompt. The program stores the input in a char array called buf . If the user enters more than 79 bytes, the resulting string and string terminator do not fit in the allocated variable.

Example 1.4

The following code segment has the possibility of a buffer overflow.

 char buf[80];     printf("Enter your first name:");     scanf("%s", buf); 

Your first thought in fixing this potential overflow might be to make buf bigger, say, 1000 bytes. What user's first name could be that long? Even if a user decides to type in a very long string of characters , 1000 bytes should be large enough to handle all but the most persistent user. However, regardless of the ultimate size that you choose, the code segment is still susceptible to a buffer overflow. The user simply needs to redirect standard input to come from an arbitrarily large file.

Example 1.5 shows a simple way to fix this problem. The format specification limits the input string to one less than the size of the variable, allowing room for the string terminator. The program reads at most 79 characters into buf but stops when it encounters a white space character. If the user enters more than 79 characters, the program reads the additional characters in subsequent input statements.

Example 1.5

The following code segment does not have a buffer overflow.

 char buf[80];       printf("Enter your first name:");       scanf("%79s", buf); 

1.6.1 Consequences of buffer overflows

To understand what happens when a buffer overflow occurs, you need to understand how programs are laid out in memory. Most program code is executed in functions with local variables that are automatic. While the details differ from machine to machine, programs generally allocate automatic variables on the program stack.

In a typical system, the stack grows from high memory to low memory. When a function is called, the lower part of the stack contains the passed parameters and the return address. Higher up on the stack (lower memory addresses) are the local automatic variables. The stack may store other values and have gaps that are not used by the program at all. One important fact is that the return address for each function call is usually stored in memory after (with larger address than) the automatic variables.

When a program writes beyond the limits of a variable on the stack, a buffer overflow occurs. The extra bytes may write over unused space, other variables, the return address or other memory not legally accessible to your program. The consequences can range from none, to a program crash and a core dump, to unpredictable behavior.

Program 1.1 shows a function that can have a buffer overflow. The checkpass function checks whether the entered string matches "mypass" and returns 1 if they match, and 0 otherwise .

Program 1.1 checkpass.c

A function that checks a password. This function is susceptible to buffer overflow .

 #include <stdio.h> #include <string.h> int checkpass(void){    int x;    char a[9];    x = 0;    fprintf(stderr,"a at %p and\nx at %p\n", (void *)a, (void *)&x);    printf("Enter a short word: ");    scanf("%s", a);    if (strcmp(a, "mypass") == 0)       x = 1;    return x; } 

Figure 1.1 shows a possible organization of the stack for a call to checkpass . The diagram assumes that integers and pointers are 4 bytes. Note that the compiler allocates 12 bytes for array a , even though the program specifies only 9 bytes, so that the system can maintain a stack pointer that is aligned on a word boundary.

Figure 1.1. Possible stack layout for the checkpass function of Program 1.1.

If the character array a is stored on the stack in lower memory than the integer x , a buffer overflow of a may change the value of x . If the user enters a word that is slightly longer than the array a , the overflow changes the value of x , but there is no other effect. Exactly how long the entered string needs to be to cause a problem depends on the system. With the memory organization of Figure 1.1, if the user enters 12 characters, the string terminator overwrites one byte of x without changing its value. If the user enters more than 12 characters, some of them overwrite x , changing its value. If the user enters 13 characters, x changes to a nonzero value and the function returns 1, no matter what characters are entered.

If the user enters a long password, the return address is overwritten, and most likely the function will try to return to a location outside the address space of the program, generating a segmentation fault and core dump. Buffer overflows that cause an application program to exit with a segmentation fault can be annoying and can cause the program to lose unsaved data. The same type of overflow in an operating system function can cause the operating system to crash.

Buffer overflows in dynamically allocated buffers or buffers with static storage can also behave unpredictably. One of our students wrote a program that appeared to show an error in the C library. He traced a segmentation fault to a call to malloc and was able to show that the program was working until the call to malloc . The program had a segmentation fault before the call to malloc returned. He eventually traced the problem to a type of buffer overflow in which the byte before a buffer dynamically allocated by a previous malloc call was overwritten. (This can easily happen if a buffer is being filled from the back and a count is off by one.) Overwriting control information stored in the heap caused the next call to malloc to crash the program.

1.6.2 Buffer overflows and security

Security problems related to buffer overflows have been known for over a decade . They first acquired national attention when on November 2, 1988, Robert Morris released a worm on the Internet. A worm is a self-replicating, self-propagating program. This program forced many system administrators to disconnect their sites from the Internet so that they would not be continually reinfected. It took several days for the Internet to return to normal. One of the methods used by the Morris worm was to exploit a buffer overflow in the finger daemon. This daemon ran on most UNIX machines to allow the display of information about users.

In response to this worm, CERT, the Computer Emergency Response Team, was created [24]. The CERT Coordination Center is a federally funded center of Internet security expertise that regularly publishes computer security alerts.

Programs that are susceptible to buffer overflow are still being written, in spite of past experiences. The first six CERT advisories in 2002 describe buffer overflow flaws in various computer systems, including Common Desktop Environment for the Sun Solaris operating environment (a windowing system), ICQ from AOL (an instant messaging program used by over 100 million users), Simple Network Management Protocol (a network management protocol used by many vendors ), and Microsoft Internet Explorer. In 1999 Steve Ballmer, the CEO of Microsoft, was quoted as saying, "You would think we could figure out how to fix buffer overflows by now." The problem is not that we do not know how to write correct code, the problem is that writing correct code takes more care than writing sloppy code. As long as priorities are to produce code quickly, sloppy code will be produced. The effects of poor coding are exacerbated by compilers and runtime systems that don't enforce range checking.

There are many ways in which buffer overflows have been used to compromise a system. Here is a possible scenario. The telnet program allows a user to remotely log in to a machine. It communicates over the network with a telnet daemon running on the remote machine. One of the functions of the telnet daemon is to query for a user name and password and then to create a shell for the user if the password is correct.

Suppose the function in the telnet daemon that requests and checks a password returns 1 if the password is correct and 0 otherwise, similar to the checkpass function of Program 1.1. Suppose the function allocates a buffer of size 100 for the password. This might seem reasonable, since passwords in UNIX are at most 8 bytes long. If the program does not check the length of the input, it might be possible to have input that writes over the return value ( x in Program 1.1), causing a shell to be created even if the password is incorrect.

Any application that runs with root privileges and is susceptible to a buffer overflow might be used to create a shell with root privileges. The implementation is technical and depends on the system, but the idea is relatively simple. First, the user compiles code to create a shell, something like the following code.

 execvl("/bin/sh", "/bin/sh", NULL);    exit(0); 

The user then edits the compiled code file so that the compiled code appears at exactly the correct relative position in the file. When the user redirects standard input to this file, the contents of the file overwrite the return address. If the bytes that overwrite the return address happen to correspond to the address of the execvl code, the function return creates a new user shell. Since the program is already running with the user ID of root, the new shell also runs with this user ID, and the ordinary user now has root privileges. The vulnerability depends on getting the bytes in the input file exactly right. Finding the address of the execvl is not as difficult as it might first appear, because most processor instruction sets support a relative addressing mode.