Introduction

Credit: David Beazley, University of Chicago

One of Python's most powerful features is its ability to be hooked to libraries and programs written in classic compiled languages such as C, C++, and Fortran. A large number of Python's built-in library modules are written as extension modules in C so that operating system services, networking functions, databases, and other features can be easily accessed from the interpreter. In addition, a number of application programmers write extensions in order to use Python as a framework for controlling large software packages coded in other languages.

The gory details of how Python interfaces with other languages can be found in various Python programming books, as well as online documentation at www.python.org (directory Demo, distributed as part of the Python source distribution, also contains several useful examples). However, the general approach revolves around the creation of special wrapper functions that hook into the interpreter. For example, if you had a C function like this:

      int gcd(int x, int y) {           int g = y;           while (x > 0) {               g = x;               x = y % x;               y = g;           }           return g;       }

and you wanted to access it from Python in a module named spam, you'd write some special wrapper code like this:

     #include "Python.h"      extern int gcd(int, int);      PyObject *wrap_gcd(PyObject *self, PyObject *args) {          int x, y, g;          if(!PyArg_ParseTuple(args, "ii", &x, &y))             return NULL;          g = gcd(x, y);          return Py_BuildValue("i", g);      }      /* List of all functions in the module */      static PyMethodDef spammethods[  ] = {         {"gcd", wrap_gcd, METH_VARARGS },         { NULL, NULL }      };      /* Module initialization function */      void initspam(void) {          Py_InitModule("spam", spammethods);      }

Once this code is compiled into an extension module, you can use the gcd function just as you would expect. For example:

>>> import spam >>> spam.gcd(63,56) 7 >>> spam.gcd(71,89) 1

This short example extends in a natural way to larger programming librarieseach function that you want to access from Python simply gets its own wrapper.

Although writing simple extension functions is fairly straightforward, writing many wrappers quickly becomes tedious and prone to error if you are building anything of reasonable complexity. Therefore, a lot of programmers rely on automatic module building tools to simplify the process. Python is fortunate to have a variety of such tools, many of which are listed below:

bgen

bgen is a module-building tool that can be found in the Tools directory of a standard Python distribution. Maintained by Jack Jansen, it is used to generate many of the extension modules available in the Macintosh version of Python, but it is not Mac specific.

pyfort

pyfort is a tool developed by Paul Dubois that can be used to build extension modules for Fortran code. Details are available at the following web page: http://pyfortran.sourceforge.net.

f2py

f2py is a wrapper generator for creating extensions in Fortran 90/95 that has been developed by Pearu Peterson. Details are available at http://cens.ioc.ee/projects/f2py2e/.

SIP

SIP is a C++ module builder developed by Phil Thompson that creates wrappers for C++ classes. The system has most notably been used to create the PyQt and PyKDE extension modules. More information can be found at http://www.thekompany.com/projects/pykde.

WrapPy

WrapPy is another C++ module builder that produces extension modules by reading C++ header files. It is developed by Greg Couch and is available at http://www.cgl.ucsf.edu/home/gregc/wrappy/index.html.

Boost Python Library

Boost Python Library, developed by David Abrahams, provides one of the most powerful and unusual C++ wrapping techniques. Classes are automatically wrapped into Python extensions by simply writing a few additional C++ classes that specify information about the extension module. More information is available at http://www.boost.org/libs/python/doc/.

SWIG

SWIG (Simplified Wrapper and Interface Generator) is an automatic extension-building tool that reads annotated C and C++ header files and produces extension modules for Python, Tcl, Perl, and a variety of other high-level languages such as Scheme, Ruby, Java, OCAML (Objective Caml), and C#. SWIG is able to wrap a large subset of C++ language features into a Python extension module. However, since I developed SWIG, I may be a little biased :-). In any event, further details are available at http://www.swig.org.

Pyrex

Pyrex is a language for writing Python extension modules, developed by Greg Ewing. The Pyrex language is a large subset of Python, with semantics slightly less fluidly dynamic than Python, and the addition of a few language constructs (particularly optional declarations of types of parameters and variables) that enables the Pyrex compiler to generate fast C code. Further details are available at http://nz.cosc.canterbury.ac.nz/~greg/python/Pyrex/.

Regardless of the approach used to build Python extension modules, certain important topics remain somewhat mysterious to many extension programmers. The recipes in this chapter describe some of the common problems and extension-building tricks that are rarely covered in the standard documentation or other Python books. Topics include interacting with threads, returning NULL values, accessing Python sequences and iterables, creating extension types, and debugging.

One recipe, in particular, highlights an especially important topic: you don't necessarily have to use other languages (even one as close to Python as Pyrex is) to write Python extensions to access functionality that's available through dynamically loaded libraries (.DLLs on Windows, .sos on Linux, .dylib on Mac OS X, etc.). It often may be sufficient to use existing third-party general-purpose extensions, such as the classic calldll or the newer ctypes packages, which enable you to wrap such dynamic libraries and make their functionality available to your Python programs, by writing just a little pure Python code.