16.8 Debugging Dynamically Loaded C Extensions with gdb

Credit: Joseph VanAndel, Michael Aivazis

16.8.1 Problem

A dynamically loaded C/C++ Python extension is giving you trouble on Unix or a Unix-like platform, and you would like to use the interactive debugger gdb to find out more about what's wrong.

16.8.2 Solution

One way to find the cause of core dumps or other serious trouble with a C Python extension is to compile the extension source with -g and then follow these steps (you may also want to recompile any other extensions you use, such as Numeric, with -g ):

% gdb /usr/bin/python2.1
(gdb) br _PyImport_LoadDynamicModule
(gdb) cont   # Repeat until your extension is loaded
(gdb) finish # to load your extension
(gdb) br wrap_myfunction  # the entry point in your code
(gdb) disable 1   # don't want to break for more modules being loaded
(gdb) continue

16.8.3 Discussion

If a dynamically loaded C/C++ extension is causing Python to core dump, or causing some other kind of serious trouble, this recipe can help you find out the root cause by showing a technique to debug your extension using gdb (if you use Unix or a Unix-like platform, and gdb is your debugger of choice). Note that the overall concept generalizes to other debuggers with abilities similar to gdb 's. You cannot set a break on your function all at once, because your function lives in a dynamic library (shared object) that isn't initially loaded. However, you can break in the PyImport_LoadDynamicModule function and eventually (when your module is at long last loaded) get control at the debugger prompt after your module is in memory. You will be able, at last, to set the breakpoint you need.

This technique works. However, if you often do this kind of thing, the process of stepping through all the modules, as Python loads them at startup, can easily become tedious . There's a handier alternative, although it's more invasive, because it requires you to modify your Python sources and rebuild Python from them.

The key idea of this handier alternative is to add a do-nothing function somewhere in the body of code that Python loads immediately. Specifically, you can edit the Modules/main.c file to include one new function:

void Py_DebugTrap(void) { }

In whatever extension you're debugging, add a call to Py_DebugTrap right where you want to break into the code. The Py_DebugTrap symbol is immediately available when you start gdb , because the symbol lives in main.c . So you can immediately set a breakpoint there when you are at the gdb prompt, then continue. This even works in parallel under MPI.

16.8.4 See Also

The gdb online documentation (just type help at the interactive prompt), manpages, and online manual (http://www.gnu.org/manual/gdb-4.17/gdb.html).

16.9 Debugging Memory Problems

Credit: Will Ware

16.9.1 Problem

You're developing C extensions, and you experience memory problems. You suspect mismanagement of reference counts and want to check whether your C extension code is managing reference counts correctly.

16.9.2 Solution

To chase these problems optimally, you need to alter Python's sources and rebuild Python. Specifically, add this function in Objects/object.c immediately before the _Py_PrintReferences function:

void
_Py_CountReferences(FILE *fp)
{
    int n;
    PyObject *op;
    for (n = 0, op = refchain._ob_next;
         op != &refchain;
         op = op->_ob_next, n += op->ob_refcnt)
    { }
    fprintf(fp, "%d refs\n", n);
}

I place in the following macros in my C extension:

#if defined(Py_DEBUG)  defined(DEBUG)
extern void _Py_CountReferences(FILE*);
#define CURIOUS(x) { fprintf(stderr, _ _FILE_ _ ":%d ", _ _LINE_ _); x; }
#else
#define CURIOUS(x)
#endif
#define MARKER(  )        CURIOUS(fprintf(stderr, "\n"))
#define DESCRIBE(x)     CURIOUS(fprintf(stderr, "  " #x "=%d\n", x))
#define DESCRIBE_HEX(x) CURIOUS(fprintf(stderr, "  " #x "=%08x\n", x))
#define COUNTREFS(  )     CURIOUS(_Py_CountReferences(stderr))

To debug, I rebuild Python using make OPT="-DPy_DEBUG" , which causes the code under Py_TRACE_REFS to be built. My own makefile for my extensions does the same trick by including these lines:

debug:
        make clean; make OPT="-g -DPy_DEBUG" all
CFLAGS = $(OPT) -fpic -O2 -I/usr/local/include -I/usr/include/python1.5

16.9.3 Discussion

If I'm developing C extensions and I run into memory problems, I find that the typical cause is mismanagement of reference counts, particularly abuses of Py_INCREF and Py_DECREF , as well as forgetfulness of the reference-count effects of functions such as Py_BuildValue , PyArg_ParseTuple , PyTuple/List_SetItem/GetItem , etc. The 1.5.2 source code base offers some help with this (search for Py_TRACE_REFS ), but I found it useful to add this recipe's function in Objects/object.c just before _Py_PrintReferences .

Unlike _Py_PrintReferences , this recipe's function will print only the total of all the reference counts in the system, so it can be used safely in loops that will repeat millions of times, whereas _Py_PrintReferences would print out way too many counts to be useful. This can help you identify errantly wandering Py_INCREF s and Py_DECREF s.

So when I suspect that one of my functions is responsible for memory problems, I liberally sprinkle the suspect function with calls to the COUNTREFS macro. This allows me to keep track of exactly how many references are being created or destroyed as I go through my function. This is particularly useful in tight loops, in which dumb mistakes can cause reference counts to grow ridiculously fast. Also, reference counts that shrink too fast (overzealous use of Py_DECREF ) can cause core dumps because the memory for objects that should still exist has been reallocated to new objects.