5.3 Smashing the Stack

This section describes a typical buffer overflow. Figure 5-4 shows an example of a stack structure after a function is called. The stack pointer points at the top of the stack, which is at the bottom in the figure.

Figure 5-4. Representation of stack structure after a function call

C++ uses the area at the top of the stack in the following order: local variables , the previous frame pointer, the return address, and the arguments of the function. This data is called the frame of the function, and it represents the status of the function. The frame pointer locates the current frame, and the previous frame pointer stores the frame pointer of the calling function.

When an attacker overflows a buffer on the stack (e.g., with extra input), the buffer will grow toward the return address. The hacker is attempting to change the return address. When the function executes, the return address is popped off the stack and the new address is executed. By overwriting this address, a hacker attempts to take control of the processor. If malicious code is located at the address, it is executed with the same privilege level as the application.