Why doesn't the weakref work on this bound method?

Question

I have a project where i\'m trying to use weakrefs with callbacks, and I don\'t understand what I\'m doing wrong. I have created simplified test that shows the exact behavio

Answer 1

You want a WeakMethod.

An explanation why your solution doesn't work can be found in the discussion of the recipe:

Normal weakref.refs to bound methods don't quite work the way one expects, because bound methods are first-class objects; weakrefs to bound methods are dead-on-arrival unless some other strong reference to the same bound method exists.

Answer 2

The other answers address the why in the original question, but either don't provide a workaround or refer to external sites.

After working through several other posts on StackExchange on this topic, many of which are marked as duplicates of this question, I finally came to a succinct workaround. When I know the nature of the object I'm dealing with, I use the weakref module; when I might instead be dealing with a bound method (as occurs in my code when using event callbacks), I now use the following WeakRef class as a direct replacement for weakref.ref(). I've tested this with Python 2.4 through and including Python 2.7, but not on Python 3.x.

class WeakRef:

    def __init__ (self, item):

        try:
            self.method   = weakref.ref (item.im_func)
            self.instance = weakref.ref (item.im_self)

        except AttributeError:
            self.reference = weakref.ref (item)

        else:
            self.reference = None


    def __call__ (self):

        if self.reference != None:
            return self.reference ()

        instance = self.instance ()

        if instance == None:
            return None

        method = self.method ()

        return getattr (instance, method.__name__)

Answer 3

According to the documentation for the Weakref module:

In the following, the term referent means the object which is referred to by a weak reference.

A weak reference to an object is not enough to keep the object alive: when the only remaining references to a referent are weak references, garbage collection is free to destroy the referent and reuse its memory for something else.

Whats happening with MyCallbackA is that you are holding a reference to it in the instances of A, thanks to -

self.MyCallbackA = MyCallbackA

Now, there is no reference to the bound method MyCallbackB in your code. It is held only in a.__class__.__dict__ as an unbound method. Basically, a bound method is created (and returned to you) when you do self.methodName. (AFAIK, a bound method works like a property -using a descriptor (read-only): at least for new style classes. I am sure, something similar i.e. w/o descriptors happens for old style classes. I'll leave it to someone more experienced to verify the claim about old style classes.) So, self.MyCallbackB dies as soon as the weakref is created, because there is no strong reference to it!

My conclusions are based on :-

import weakref

#Trace is called when the object is deleted! - see weakref docs.
def trace(x):
    print "Del MycallbackB"

class A(object):
    def __init__(self):

        def MyCallbackA():
            print 'MyCallbackA'
        self.MyCallbackA = MyCallbackA
        self._testA = weakref.proxy(self.MyCallbackA)
        print "Create MyCallbackB"
        # To fix it, do -
        # self.MyCallbackB = self.MyCallBackB
        # The name on the LHS could be anything, even foo!
        self._testB = weakref.proxy(self.MyCallbackB, trace)
        print "Done playing with MyCallbackB"

    def MyCallbackB(self):
        print 'MyCallbackB'

    def test_a(self):
        self._testA()

    def test_b(self):
        self._testB()

if __name__ == '__main__':
    a = A()  
    #print a.__class__.__dict__["MyCallbackB"] 
    a.test_a()

Output

Create MyCallbackB
Del MycallbackB
Done playing with MyCallbackB
MyCallbackA

Note :
I tried verifying this for old style classes. It turned out that "print a.test_a.__get__" outputs -

<method-wrapper '__get__' of instancemethod object at 0xb7d7ffcc>

for both new and old style classes. So it may not really be a descriptor, just something descriptor-like. In any case, the point is that a bound-method object is created when you acces an instance method through self, and unless you maintain a strong reference to it, it will be deleted.