Recipe7.5.Holding Bound Methods in a Picklable Way

Recipe 7.5. Holding Bound Methods in a Picklable Way

Credit: Peter Cogolo

Problem

You need to pickle an object, but that object holds (as an attribute or item) a bound method of another object, and bound methods are not picklable.

Solution

Say you have the following objects:

import cPickle class Greeter(object):     def _ _init_ _(self, name):         self.name = name     def greet(self):         print 'hello', self.name class Repeater(object):     def _ _init_ _(self, greeter):         self.greeter = greeter     def greet(self):         self.greeter( )         self.greeter( ) r = Repeater(Greeter('world').greet)

Were it not for the fact that r holds a bound method as its greeter attribute, you could pickle r very simply:

s = cPickle.dumps(r)

However, upon encountering the bound method, this call to cPickle.dumps raises a TypeError. One simple solution is to have each instance of class Repeater hold, not a bound method directly, but rather a picklable wrapper to it. For example:

class picklable_boundmethod(object):     def _ _init_ _(self, mt):         self.mt = mt     def _ _getstate_ _(self):         return self.mt.im_self, self.mt.im_func._ _name_ _     def _ _setstate_ _(self, (s,fn)):         self.mt = getattr(s, fn)     def _ _call_ _(self, *a, **kw):         return self.mt(*a, **kw)

Now, changing Repeater._ _init_ _'s body to self.greeter = picklable_boundmethod(greeter) makes the previous snippet work.

Discussion

The Python Standard Library pickle module (just like its faster equivalent cousin cPickle) pickles functions and classes by namethis implies, in particular, that only functions defined at the top level of a module can be pickled (the pickling of such a function, in practice, contains just the names of the module and function).

If you have a graph of objects that hold each other, not directly, but via one another's bound methods (which is often a good idea in Python), this limitation can make the whole graph unpicklable. One solution might be to teach pickle how to serialize bound methods, along the same lines as described in Recipe 7.6. Another possible solution is to define appropriate _ _getstate_ _ and _ _setstate_ _ methods to turn bound methods into something picklable at dump time and rebuild them at load time, along the lines described in Recipe 7.4. However, this latter possibility is not a good factorization when you have several classes whose instances hold bound methods.

This recipe pursues a simpler idea, based on holding bound methods, not directly, but via the picklable_boundmethod wrapper class. picklable_boundmethod is written under the assumption that the only thing you usually do with a bound method is to call it, so it only delegates _ _call_ _ functionality specifically. (You could, in addition, also use _ _getattr_ _, in order to delegate other attribute accesses.)

In normal operation, the fact that you're holding an instance of picklable_boundmethod rather than holding the bound method object directly is essentially transparent. When pickling time comes, special method _ _getstate_ _ of picklable_boundmethod comes into play, as previously covered in Recipe 7.4. In the case of picklable_boundmethod, _ _getstate_ _ returns the object to which the bound method belongs and the function name of the bound method. Later, at unpickling time, _ _setstate_ _ recovers an equivalent bound method from the reconstructed object by using the getattr built-in for that name. This approach isn't infallible because an object might hold its methods under assumed names (different from the real function names of the methods). However, assuming you're not specifically doing something weird for the specific purpose of breaking picklable_boundmethod's functionality, you shouldn't ever run into this kind of obscure problem!

Recipe7.5.Holding Bound Methods in a Picklable Way