Antihacking Techniques

Most frequently, buffer overflow is concentrated in strictly defined locations, such as the following functions: strcpy , strcat, gets, sprintf , the family of scanf functions, [v] [f]printf, [v] snprintf , and syslog . In nine cases out often, control is passed to the shellcode by replacing the return address from a function. Other methods fall on modification of indexes, pointers, and other types of variables . At the same time, buffer overflow usually is sequential, which means that the continuous memory region is overwritten. In the course of index overflow, several memory cells far after the end of the buffer are overwritten. It occurs only occasionally and isn't considered a serious threat.

This narrows the range of " suspects " and considerably simplifies the task of controlling buffers. There are lots of utilities preventing (or at least trying to prevent) overflow errors. In the next few sections, I'll consider some of them.

StackGuard

This is probably the most successful and most popular antihacker protector. It presents a patch for GCC that modifies the machine code of the function prologue ( function_prologue ) and function epilogue ( function_epilogue ) inserted by the compiler into the beginning and into the end of every function. When entering the function, a sensible indicator (also known as the canary word) is set above the return address. The hacker in the course of sequential overflow inevitably overwrites the canary word. Before exiting the function, the canary word is compared with the original, stored far from the copy in a location unreachable by the hacker. If integrity of the canary word is violated, the program reports that it has been hacked and stops operation (thus, making a DoS attack on itself).

To prevent the forgery of the canary word, StackGuard takes a range of precautions . The canary word is the combination of three terminating characters ( 0x00000000L, CR, Lf , and FFh ), which by most functions are interpreted as terminators of input, with an arbitrary tie-in, which is read from the /dev/urandom device or generated on the basis of the current time (if /dev/urandom is not available). This technique protects only against sequential overflow errors (but not against all of them). However, it cannot protect against index overflow.

When necessary, StackGuard can disable modification of the return address for the time of function execution. This improves the protection greatly but drops the performance (the canary word has practically no negative effect on the performance). In addition, to implement this mechanism a certain level of support at the kernel level is required, and most kernels do not provide such support.

StackGuard can be downloaded from the site ftp://ftp. ibiblio .org/pub/Linux/distributions/immunix/ .

Nonexecutable Stack

This is a special patch from Solar Designer built into the Linux kernel, thus making the stack nonexecutable. Overflowing buffers will continue to result in application crash, but it becomes impossible to directly pass control to the shellcode. Well, not impossible , but exceptionally difficult to implement (for more details, read the " Defeating Solar Designer's Nonexecutable Stack Patch " article, which is available at http://www. insecure .org/sploits/non-executable.stack.problems.html ).

This doesn't reduce the performance and doesn't require recompiling of the existing application. However, this patch cannot pretend to play the role of a universal solution. Patches are available only for older kernel versions (2.0, 2.2, and 2.4). Furthermore, there are lots of possible conflicts. Nevertheless, it is not expedient to abandon the idea of nonexecutable stack altogether.

The patch can be downloaded from http://www.openwall.com/linux/ .

ITS4 Software Security Tool

This is a static analyzer of the source code, aiming at searching for overflowing buffers and some other errors. It notices the calls to potentially dangerous functions, such as strcpy/ memcpy , and carries out superficial semantic analysis, trying to assess the potential danger. It also produces advisories on the improvement of potentially dangerous code (although in most cases these advisories are either self-evident or openly foolish). It supports both C and C++ dialects. This is a command-line utility working under both Windows and UNIX.

To download the ITS4 software security tool, hit http://www. cigital .com/its4/ .

Flawfinder

This is a simple static analyzer of the source code written in C or C++. It tries to detect overflow errors, but it does this rather clumsily. Instead of semantic analysis of the code, it carries out a simple pattern search. Flawfinder notices only the function name ( strcpy, strcat , etc.) and the arguments passed to it (constant, string, or pointer to a buffer), evaluating potential danger in conventional "hits." Nevertheless, it is useful for obtaining general information about the program, especially if this is a program written by someone else.

Flawfinder is available at the following address: http://www.dwheeler.com/flawfinder/ .