Recipe6.10.Keeping References to Bound Methods Without Inhibiting Garbage Collection

Recipe 6.10. Keeping References to Bound Methods Without Inhibiting Garbage Collection

Credit: Joseph A. Knapka, Frédéric Jolliton, Nicodemus

Problem

You want to hold references to bound methods, while still allowing the associated object to be garbage-collected.

Solution

Weak references (i.e., references that indicate an object as long as that object is alive but don't keep that object alive if there are no other, normal references to it) are an important tool in some advanced programming situations. The weakref module in the Python Standard Library lets you use weak references.

However, weakref's functionality cannot directly be used for bound methods unless you take some precautions. To allow an object to be garbage-collected despite outstanding references to its bound methods, you need some wrappers. Put the following code in a file named weakmethod.py in some directory on your Python sys.path:

import weakref, new class ref(object):     """ Wraps any callable, most importantly a bound method, in         a way that allows a bound method's object to be GC'ed, while         providing the same interface as a normal weak reference. """     def _ _init_ _(self, fn):         try:             # try getting object, function, and class             o, f, c = fn.im_self, fn.im_func, fn.im_class         except AttributeError:                # It's not a bound method             self._obj = None             self._func = fn             self._clas = None         else:                                 # It is a bound method             if o is None: self._obj = None    # ...actually UN-bound             else: self._obj = weakref.ref(o)  # ...really bound             self._func = f             self._clas = c     def _ _call_ _(self):         if self.obj is None: return self._func         elif self._obj( ) is None: return None         return new.instancemethod(self._func, self.obj( ), self._clas)

Discussion

A normal bound method holds a strong reference to the bound method's object. That means that the object can't be garbage-collected until the bound method is disposed of:

>>> class C(object): ...     def f(self): ...         print "Hello" ...     def _ _del_ _(self): ...         print "C dying" ... >>> c = C( )  >>> cf = c.f >>> del c      # c continues to wander about with glazed eyes... >>> del cf     # ...until we stake its bound method, only then it goes away: C dying

This behavior is most often handy, but sometimes it's not what you want. For example, if you're implementing an event-dispatch system, it might not be desirable for the mere presence of an event handler (i.e., a bound method) to prevent the associated object from being reclaimed. The instinctive idea should then be to use weak references. However, a normal weakref.ref to a bound method doesn't quite work the way one might expect, because bound methods are first-class objects. Weak references to bound methods are dead-on-arrivalthat is, they always return None when dereferenced, unless another strong reference to the same bound-method object exists.

For example, the following code, based on the weakref module from the Python Standard Library, doesn't print "Hello" but raises an exception instead:

>>> import weakref >>> c = C( ) >>> cf = weakref.ref(c.f) >>> cf         # Oops, better try the lightning again, Igor... <weakref at 80ce394; dead> >>> cf( )( ) Traceback (most recent call last): File "", line 1, in ? TypeError: object of type 'None' is not callable

On the other hand, the class ref in the weakmethod module shown in this recipe allows you to have weak references to bound methods in a useful way:

>>> import weakmethod >>> cf = weakmethod.ref(c.f) >>> cf( )( )     # It LIVES! Bwahahahaha! Hello >>> del c      # ...and it dies C dying >>> print cf( ) None

Calling the weakmethod.ref instance, which refers to a bound method, has the same semantics as calling a weakref.ref instance that refers to, say, a function object: if the referent has died, it returns None; otherwise, it returns the referent. Actually, in this case, it returns a freshly minted new.instancemethod (holding a strong reference to the objectso, be sure not to hold on to that, unless you do want to keep the object alive for a while!).

Note that the recipe is carefully coded so you can wrap into a ref instance any callable you want, be it a method (bound or unbound), a function, whatever; the weak references semantics, however, are provided only when you're wrapping a bound method; otherwise, ref acts as a normal (strong) reference, holding the callable alive. This basically lets you use ref for wrapping arbitrary callables without needing to check for special cases.

If you want semantics closer to that of a weakref.proxy, they're easy to implement, for example by subclassing the ref class given in this recipe. When you call a proxy, the proxy calls the referent with the same arguments. If the referent's object no longer lives, then weakref.ReferenceError gets raised instead. Here's an implementation of such a proxy class:

class proxy(ref):     def _ _call_ _(self, *args, **kwargs):         func = ref._ _call_ _(self)         if func is None:             raise weakref.ReferenceError('referent object is dead')         else:             return func(*args, **kwargs)     def _ _eq_ _(self, other):         if type(other) != type(self):             return False         return ref._ _call_ _(self) == ref._ _call_ _(other)

Recipe6.10.Keeping References to Bound Methods Without Inhibiting Garbage Collection