Chapter 9: Methods of Protecting Buffers Against Overflow

 Download CD Content

Overview

The time when compilers were compiling and programmers were programming has long gone. Contemporary compilers are powerful tools combining the functionality and ease of use of a kitchen appliance with the speed of a dive-bomber. When a compiler's functional capabilities become insufficient, the programmer might resort to lots of useful utilities. Such tools are available in such abundance that sometimes they begin flashing before your eyes. How do you choose the most useful tools that will actually be helpful?

C/C++ programming languages are not intended for application programmers. These languages are different from Pascal, which concatenates strings in the same way as other data types. They are far from Ada, with its support for dynamic arrays and built-in boundary control. The ideology of C is the best expressed by the words of Japanese animator Hayao Miyazaki: "Why use the computer for doing things that can be done manually?" The C programmer must make all checks on his or her own. If the programmer forgets this once (or simply makes an error because of carelessness), the consequences in the form of instable operation, worms, or memory leaks won't make users wait long.

At first glance, it might seem that individuals who program in C poorly might prefer any other language, such as Java or Fortran. But they do not want to do so. They will reproach the developers of the C language, but they will never abandon C. Attempts at improving the language by adding such features as a garbage collector have been undertaken multiple times. Java appeared as the result of one of such projects, and most programmers that had to face it will remember doing so for a long time. It was slow, inconvenient, and unsafe despite all improvements (let the many fans of this programming language forgive me; it isn't as bad as I have just said, but tastes do differ ).

Some friendly programmers suggest not touching the languages, leaving C/C++ as they are, but changing the compiler to make it insert the required check code after each potentially dangerous operation. Others suggest rewriting all standard libraries to make them recognize typical memory allocation errors. However, this might be achieved only at the expense of considerable performance degradation, which is too bad.

Static analyzers carry out all checks before compiling, drawing the programmer's attention to all suspicious locations that might become the sources of problems. After that, the programmer must correct the problems on his or her own. Unfortunately, the possibilities of static analyzers are limited, and many errors remain unnoticed.

In other words, things look bad. The best way to eliminate errors is to use your own brain, along with the compiler and the various plug-ins and auxiliary tools for it as an additional protection level. If this works, you are lucky.

In this chapter, additional tools for the compiler will be considered . They can be classified into two categories ” tools for antihacker protection, which prevent buffer overflow errors (and uploading of the shellcode), and detectors of memory-allocation errors, which prevent the program from crashing or taking unpredictable actions.

The number of additional tools for GNU C Compiler (GCC) and other compilers grows steadily, and new tools appear every day. Some of them don't become popular, but some become integral parts of GCC. Thus, plug-ins that yesterday were separate projects might tomorrow be integrated into GCC. Because of this, before looking for the required utility on the Internet, it makes sense to find out whether or not the compiler contains a similar feature. Distribution sets are another good source of additional features. In particular, BSD contains dmalloc and even boehm-gc ” an automated garbage collector for C/C++ from Hewlett-Packard. It moves as slowly as a snail , but it works! The main problem is preventing the programming process from turning into a chase after new features because, after all, programming is a creative process, during which the programmer uses his or her own head and hands.