Problems with Memory Allocation

Problems with memory allocation are mainly related to classical C. In C++, there are lots of mechanisms for solving such problems. Constructors and destructors, overloaded operators, and objects with a limited visibility zone eliminate part of the errors characteristic for a programmer who allocates memory and forgets to release it. On the other hand, more sophisticated semantics of C++ considerably complicate the static analysis, forcing programmers to resort to run-time control, which is carried out directly at run time. Run-time control slightly reduces the performance.

Debug versions of libraries that have strict control over the dynamic memory (also known as the heap) are popular. They detect lots of errors that are difficult to find and reproduce, which earlier could be eliminated only by spending several days on debugging, without breaks for sleep or snacks. Because of performance considerations, they are used only at the stages of development and alpha testing and are excluded from the release version.

CCured

This is a C tool that protects the program against problems with memory allocation (going beyond the buffer limits, using uninitialized pointers, and so on). It works according to the principle of the sourcelsource translator, which accepts raw source code and inserts additional checks in suspicious locations. Thus, instead of correcting errors, it simply hammers them deeper and deeper. This protector is a kind of safety valve, which prevents the program from crashing and thwarts some remote attacks based on the insertion of shellcode. Nevertheless, the intruder still can organize DoS attacks. Furthermore, additional checks considerably reduce the performance of the protected program (the performance drop is 10% to 60%, depending on the quality of the source code).

Small programs are translated automatically. However, in serious projects programmers have to carefully rework the text produced by CCured, because it might spoil the program instead of correcting it. Nevertheless, the process of correcting the protected listing is described in details. The developers of CCured have managed to digest the source code of sendmail, bind, openssl , Apache, and other applications, spending several days on each. In addition, run-time control implemented in CCured, is more reliable than static analysis.

CCured can be downloaded from http://manju.cs.berkeley.edu/ccured/ .

Memwatch

This is the set of debug functions for detecting memory-allocation errors in programs supplied with source code. It is made up of the memwatch.h header file and the memwatch.c kernel, written in ANSI C. This ensures compatibility of all "normal" compilers and platforms (the developers declare the support for PC-lint 7.0k, Microsoft Visual C++ 16- and 32-bit versions, Microsoft C for MS-DOS, SAS C for Amiga 500, GCC, and some other platforms and compilers). C++ support is in embryonal state.

Standard functions for memory allocation ( malloc, realloc, free ) are "wrapped" into the debug "wrapper," which traces memory leaks, double release of the pointers, access to uninitialized pointers, and exit beyond the allocated memory block. Also, some watch blocks are created, intended to trace stray pointers accessing unallocated memory areas. All detected errors are logged. Macros such as ASSET and VERIFY are replaced by their advanced versions, which instead of immediate termination of the malfunctioning program allow the user the standard set of possible actions: Abort, Retry , and Ignore .

The platform-dependent part of the code is not implemented by the developers; therefore, programmers must write functions like mwIsReadAddr/mwIsSafeAddr on their own. Another serious drawback is that the program must be explicitly prepared for working with Memwatch, which in some cases is unacceptable. Multithreading support is in the embryonal state, and it is impossible to predict when its fully functional support will be implemented.

Memwatch is downloadable from http://www.linkdata.se/sourcecode.html .

Dmalloc , the Debug Malloc Library

This is the debug version of the library for working with memory, replacing such built-in C functions as malloc, realloc, calloc , and free . At the same time, there is no need to modify the source code of the application (although, if desired, it is possible to carry out explicit memory checks).

The service provided by dmalloc is standard for utilities of this class: It detects memory leaks; goes beyond the buffer boundaries, statistics, and logging; and specifies line numbers and file names . If memory checks are included for each operation, the resulting application becomes horribly slow so that at least a Pentium-4/Prescott processor is required.

The dmalloc library works practically everywhere: AIX, BSD/OS, DG/UX, Free/Net/OpenBSD, GNU/Hurd, HP-UX, IRIX, Linux, MS-DOS, NeXT, OSF, SCO, Solaris, SunOS, Ultrix, Unixware, Windows, and even the Unices operating system on the CrayTSE supercomputer. It requires sophisticated configuration procedures and preliminary steps to be carried out, which cannot be achieved without previously reading the documentation. Naturally, this is a drawback. On the other hand, it provides fully functional support for multithreading, which is certainly a significant advantage in comparison to most of its competitors .

The dmalloc library can be downloaded from http://dmalloc.com/ .

Checker

This is another debug library offering custom implementation of the malloc, realloc , and free function. It displays error messages any time free or realloc accept the pointer obtained from a source other than malloc , and it traces repeated attempts at releasing the pointers that have already been released, as well as attempts at accessing uninitialized memory areas. It also retards actual release of the memory blocks for some time, during which it vigilantly tracks whether any attempts at accessing them take place. This debug library contains a garbage detector, which is called either from the debugger or directly from the program being investigated. In general, it is implemented simply but tastefully. An additional advantage is that this debug library has practically no negative effect on system performance. It works well with the GNU compiler. (As relates to other compilers, I didn't check them.)

Checker can be downloaded from http://www.gnu.org/software/checker/checker.html .

A Simple Macro for Detecting Memory Leaks

Listing 9.1 shows a simple macro that can be used for detecting memory leaks.

Listing 9.1: A wrapper for malloc

 #ifdef DEBUG #define MALLOC(ptr, size) do { \ ptr = malloc (size); \ pthread_mutex_lock(&gMemMutex); \ gMemCounter++; \ pthread_mutex_unlock(&gMemMutex); \ }while (0) #else #define MALLOC(ptr,size) ptr  =  malloc (size) #endif 

 

This macro is intended to be used instead of the standard malloc function (you won't have any problems writing a similar macro for the free function). If, after exiting the program, gMemCounter has a nonzero value, this means that there is a memory leak somewhere in the program. Generally, an inverse statement is not true. Memory that has not been released can be combined with a double call for the free function, and gMemCounter will be equal to zero as a result. However, the problem won't be eliminated. An "extra" do/while loop is intended for bypassing the constructs that appear as follows : if(xxx) MALLOC (p, s); else yyy; . It will be possible to do without it, but in this case it will be necessary to manually insert braces.

Handling Memory Allocation Errors

Constantly checking if memory allocation was successful is tedious . Furthermore, it clutters the source code and results in unjustified growth in the size of the compiled program and considerable overhead (Fig. 9.1). This solution is not the best one (Listing 9.2)

Figure 9.1: Overhead for the explicit check of the buffer boundaries before each function call

Listing 9.2: An example of poor implementation of memory-allocation success or failure

 char *p; p = malloc(BLOCK_SIZE); if (!p) {        fprintf(stderr,        "-ERR: Insufficient memory to continue the operation\n");        _exit(); } 

 

One of the possible solutions to this problem is creating a wrapper around the intensely-used function. This wrapper would check whether the function has completed successfully, display an error message if required, and then terminate the program and pass control to the appropriate handler of this situation (Listing 9.3).

Listing 9.3: An improved variant of the implementation of the memory-allocation check

 void* my_malloc(int x) {         int *z;         z = malloc(x);         if (!z) GlobalError_and_Save_all_Unsaved_Data; } 

 

Eliminating Heap Fragmentation

It is possible to go even further and make the MyMalloc function return the pointer to the pointer. This will allow defragmentation of the heap, provided that block addressing will always be carried out through the base pointer, which will be constantly re-read from the memory. To prevent the compiler from cashing it into a register, the pointer must be declared volatile. As a variant, it is possible to protect the program by critical sections to prevent the memory block from being moved in the course of working with it. Both approaches reduce the performance, and, because of this, they do not appear as undisputable solutions.

A Poor Check Is Worse than Nothing

A code like p = malloc(x) ; if (!p) return 0; often is even worse than no check. This is because when accessing the zero pointer (corresponding to the memory-allocation error), the operating system would complain and inform the user about the problem cause and the address, at which the error occurred. The clumsy check, instead of doing this, will silently terminate the program, making everyone wonder what happened .

Emergency Stock or Memory Reserve

Virtual memory of the system tends to come to an end unexpectedly. If this happens, an attempt at allocating a new memory block using malloc results in an error. In such a case, it is important to correctly save all unsaved data and close the program. What could you do if to save the data a small memory block is needed? It's easy! When the program starts, use the malloc function to allocate the memory block of the size required to correctly close the application, saving all unsaved data in case of a memory shortage. This memory reserve should be used only in an emergency.