Hack 73 Use the GNU Debugger to Analyze a Buffer Overflow

You don't have to be a programmer to use a debugger.

As an end user, you may not realize that you have the ability to analyze security exploits. After all, the organization that distributes your operating system of choice or the provider of a given application will deal with security issues and make updates available.

However, keep in mind that Security Officers apply the same tools and techniques that end users use for debugging programs. Knowing how to analyze a problem will help you to troubleshoot any misbehaving process in a Unix environment.

7.6.1 An Example Exploit

Analyzing a malfunctioning process starts with basic information, such as error messages and return values. Sometimes those aren't enough, though. Some error messages are unclear. In the case of security vulnerabilities, there may not be an error code or return value, because the program may crash or misbehave silently.

The BSDs provide several tools to analyze a program's execution. You can monitor system calls with ktrace and resources with fstat. You can run a debugger such as GDB, the GNU Debugger, and watch your operating system's internal operation.

In some cases, a program must run in a particular environment, which may make it difficult to analyze due to the limitations of some tools. For example, a telnetd advisory from 2001 (http://www.cert.org/advisories/CA-2001-21.html) affected most Unix operating systems. This particular vulnerability came to light when a group called TESO released an example exploit for it.

On Unix systems, telnetd runs as root, so that once the system authenticates the user, the process has the privileges required to set the user ID of the login shell to that of the user who logged in. This means that a remote entity who can cause telnetd to misbehave by sending it carefully designed input could execute processes as root on your system.

On most Unix systems, telnetd does not run as a standalone daemon. Since logins are relatively infrequent (on the system timescale compared to thousands of interrupts per second), the inetd service starts telnetd as needed.

This is a simple example of the data stream sufficient to crash vulnerable telnetds using perl and nc (netcat):

% perl -e 'print "\377\366"x512' |  nc testhost telnet

This was the example I used to diagnose the problem and test the fix. If you run this command against an impervious Telnet daemon, you'll see the following output:

% perl -e 'print "\377\366"x512' | nc testhost telnet [Yes] [Yes] [Yes]

The [Yes] message will repeat 512 times because the characters you sent, \377\366, represent the Telnet protocol's "ARE YOU THERE" control message, and you asked the question 512 times.

If you run this command against a vulnerable telnetd, the output can vary. In some cases, your connection may close before you get 512 [Yes] responses because telnetd crashed. In other cases, you may receive seemingly random output from portions of the telnetd memory space. These both indicate that the program did something it was not supposed to, due to the specific input you gave it.

7.6.2 Using the GNU Debugger

In order to fix the problem, we need to find out where the executable did something incorrectly. We would like to run the program under the control of GDB, but we cannot start telnetd from the command line the way we usually would when debugging most executables. Normally, GDB is invoked in one of three ways.

First, to run a program and debug it, type:

% gdb  programname GNU gdb 5.3nb1 Copyright 2002 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you  are welcome to change it and/or distribute copies of it under certain  conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB.  Type "show warranty" for details. This GDB was configured as "i386--netbsdelf"...(no debugging symbols found)... (gdb) run

If this is your first time using gdb, type help at the (gdb) prompt. Type quit when you are finished using the debugger.

Second, to examine the core file of a program that has already crashed, use:

% gdb  programname   programname .core

Third, to examine a program that is already running, type:

% gdb  programname   processid

In the case of telnetd, we cannot use the first method, because inetd must start telnetd in order to attach it to a network socket and operate properly. We cannot use the second method, because processes that run with root privileges do not leave core files, since the program's memory image could contain sensitive data.

That leaves the third method. Attaching to a running process is problematic because telnetd isn't running until someone connects. We'll need to modify our attack script:

% perl -e 'sleep 30; print "\377\366"x512' |  nc testhost telnet

Now nc opens a socket to the testhost, inetd spawns a telnetd in response, and perl waits for 30 seconds before sending the attack string.

In another terminal, on the testhost, we say:

% ps -ax | grep telnetd 27857 ??  S      0:00.05 telnetd 27859 pd  S+     0:00.02 grep telnetd % gdb /usr/libexec/telnetd 27857 GNU gdb[...] Attaching to program `/usr/libexec/telnetd', process 27857

From here we can allow telnetd to crash and observe the exact type of error that caused the crash. If we've built telnetd with debugging information, GDB will even display the line of source code the program was executing when it crashed. Now we can use our favorite debugging techniques and either insert debugging messages or use GDB and set breakpoints and watchpoints to discover at what point the program went off course. We can then determine what changes to make to correct the error and prevent the exploit.

If you're not a programmer, you can save the information and send it to the developers.

7.6.3 Hacking the Hack

We were fortunate in this example because we had details of the exploit. That made it easy to experiment and try different approaches. In many cases, however, you won't know the details of an exploit, and you may only know that there is a problem because of error messages in your logs.

You can use tcpdump to capture the traffic on the relevant port. Once you can correlate the timestamp of the log's error message with some of your tcpdump traffic, you can take the data sent in an attack and create a Perl script to resend it. You can then apply the techniques already described to analyze and correct the problem.

7.6.4 See Also

• man ktrace

• man fstat

• man gdb

• The Netcat web site; see the Read Me file (http://www.atstake.com/research/tools/network_utilities)

• The "Debugging with GDB" tutorial (http://www.delorie.com/gnu/docs/gdb/gdb_toc.html)