Recipe17.5.Using SWIG-Generated Modules in a Multithreaded Environment

Recipe 17.5. Using SWIG-Generated Modules in a Multithreaded Environment

Credit: Joe VanAndel, Mark Hammond

Problem

You want to use SWIG-generated modules in a multithreaded environment; therefore, the C code in those modules must release the Python global interpreter lock (see the Introduction to Chapter 9 for more information about the global interpreter lock).

Solution

Use a typemap for SWIG, written by Mark Hammond, that was posted on comp.lang.python. It maps Win32 API functions that return BOOL to Python functions that return None and raise exceptions to diagnose errors. The wrapped function must set the standard Windows global LastError if it returns FALSE (indicating that it has detected an error). The wrapping function also automatically releases the Python global interpreter lock (GIL) for the duration of the wrapped function's execution, to allow free multithreading.

%typedef BOOL BOOLAPI %typemap(python,except) BOOLAPI {       Py_BEGIN_ALLOW_THREADS       $function         Py_END_ALLOW_THREADS         if (!$source)  {               $cleanup                return PyWin_SetAPIError("$name");         } }

Discussion

To use multiple threads effectively, you must release the Python GIL from your C-coded extension whenever it's safe to do so. The simplest way to do this with SWIG is to use an except directive, as shown in the recipe's typemap. Within the typemap, you can then use the normal Python C API's macros Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS (around the call to the wrapped function, indicated by the special SWIG directive $function) to release the GIL and acquire it again.

Another interesting effect of this simple typemap is that it turns the C-oriented error-return convention (returning FALSE and setting a global error indicator code) into a highly Pythonic convention (raising an exception).

See Also

SWIG and its typemaps are documented at http://www.swig.org; Windows API documentation on LastError is available from the Microsoft MSDN site at http://msdn.microsoft.com; Chapter 9 for general information on threads and particularly its Introduction for information on the GIL.