Nesting descriptors/decorators in python

Question

I'm having a hard time understanding what happens when I try to nest descriptors/decorators. I'm using python 2.7.

For example, let's take the following simplified versions of property and classmethod:

class MyProperty(object):
    def __init__(self, fget):
        self.fget = fget
    def __get__(self, obj, objtype=None):
        print 'IN MyProperty.__get__'
        return self.fget(obj)

class MyClassMethod(object):
    def __init__(self, f):
       self.f = f
    def __get__(self, obj, objtype=None):
        print 'IN MyClassMethod.__get__'
        def f(*args, **kwargs):
            return self.f(objtype, *args, **kwargs)
        return f

Trying to nest them:

class A(object):
    # doesn't work:
    @MyProperty
    @MyClassMethod
    def klsproperty(cls):
        return 555
    # works:
    @MyProperty
    def prop(self):
        return 111
    # works:
    @MyClassMethod
    def klsmethod(cls, x):
        return x**2

% print A.klsproperty
IN MyProperty.__get__
...
TypeError: 'MyClassMethod' object is not callable

The __get__ method of the inner descriptor MyClassMethod is not getting called. Failing to figure out why, I tried throwing in (what I think is) a no-op descriptor:

class NoopDescriptor(object):
    def __init__(self, f):
       self.f = f
    def __get__(self, obj, objtype=None):
        print 'IN NoopDescriptor.__get__'
        return self.f.__get__(obj, objtype=objtype)

Trying to use the no-op descriptor/decorator in nesting:

class B(object):
    # works:
    @NoopDescriptor
    @MyProperty
    def prop1(self):
        return 888
    # doesn't work:
    @MyProperty
    @NoopDescriptor
    def prop2(self):
        return 999

% print B().prop1
IN NoopDescriptor.__get__
IN MyProperty.__get__
888
% print B().prop2
IN MyProperty.__get__
...
TypeError: 'NoopDescriptor' object is not callable

I don't understand why B().prop1 works and B().prop2 does not.

Questions:

1. What am I doing wrong? Why am I getting a object is not callable error?
2. What's the right way? e.g. what is the best way to define MyClassProperty while re-using MyClassMethod and MyProperty (or classmethod and property)

Answer 1

You can make your code work if you make MyProperty apply the descriptor protocol to its wrapped object:

class MyProperty(object):
    def __init__(self, fget):
        self.fget = fget
    def __get__(self, obj, objtype=None):
        print('IN MyProperty.__get__')
        try:
            return self.fget.__get__(obj, objtype)()
        except AttributeError: # self.fget has no __get__ method
            return self.fget(obj)

Now your example code works:

class A(object):
    @MyProperty
    @MyClassMethod
    def klsproperty(cls):
        return 555

print(A.klsproperty)

The output is:

IN MyProperty.__get__
IN MyClassMethod.__get__
555

Answer 2

In this case, when the decorators are used without parameters, a decorator is called with the function it decorates as its parameter. The decorator's return value is used instead of the decorated function. So:

@MyProperty
def prop(self):
    ...

is equivalent to:

def prop(self):
    ...
prop = MyProperty(prop)

Since MyProperty implements the descriptor protocol, accessing A.prop will actually call A.prop.__get__(), and you've defined __get__ to call the object which was decorated (in this case, the original function/method), so everything works fine.

Now, in the nested case:

@MyProperty
@MyClassMethod
def prop(self):
    ...

The equivalent is:

def prop(self):
    ...
prop = MyClassMethod(prop)   # prop is now instance of MyClassMethod
prop = MyProperty(prop)      # prop is now instance of MyProperty
                             # (with fget == MyClassMethod instance)

Now, as before, accessing A.prop will actually call A.prop.__get__() (in MyProperty) which then tries to call the instance of MyClassMethod (the object which was decorated and stored in the fget attribute).

But the MyClassMethod does not have a __call__ method defined, so you get the error MyClassMethod is not callable.

---

And to address your second question: A property is already a class attribute - in your example, accessing A.prop will return the value of the property in the class object and A().prop will return the value of the property in an instance object (which can be the same as the class object if the instance did not override it).

Answer 3

I found the definitive answer to my own old question in Graham Dumpleton's fascinating blog.

In short, the decorators I wrote do not honour the descriptors protocol, by trying to call the wrapped function/object directly, instead of first giving them a chance to perform their "descriptor magic" (by calling their __get__() first).