Python property descriptor design: why copy rather than mutate?

Question

I was looking at how Python implements the property descriptor internally. According to the docs property() is implemented in terms of the descriptor protocol, repr

Answer 1

Let's start with a bit of history, because the original implementation had been equivalent to your alternative (equivalent because property is implemented in C in CPython so the getter, etc. are written in C not "plain Python").

However it was reported as issue (1620) on the Python bug tracker back in 2007:

As reported by Duncan Booth at http://permalink.gmane.org/gmane.comp.python.general/551183 the new @spam.getter syntax modifies the property in place but it should create a new one.

The patch is the first draft of a fix. I've to write unit tests to verify the patch. It copies the property and as a bonus grabs the __doc__ string from the getter if the doc string initially came from the getter as well.

Unfortunately the link doesn't go anywhere (I really don't know why it's called a "permalink" ...). It was classified as bug and changed to the current form (see this patch or the corresponding Github commit (but it's a combination of several patches)). In case you don't want to follow the link the change was:

 PyObject *
 property_getter(PyObject *self, PyObject *getter)
 {
-   Py_XDECREF(((propertyobject *)self)->prop_get);
-   if (getter == Py_None)
-       getter = NULL;
-   Py_XINCREF(getter);
-   ((propertyobject *)self)->prop_get = getter;
-   Py_INCREF(self);
-   return self;
+   return property_copy(self, getter, NULL, NULL, NULL);
 }

And similar for setter and deleter. If you don't know C the important lines are:

((propertyobject *)self)->prop_get = getter;

and

return self;

the rest is mostly "Python C API boilerplate". However these two lines are equivalent to your:

self.fget = fget
return self

And it was changed to:

return property_copy(self, getter, NULL, NULL, NULL);

which essentially does:

return type(self)(fget, self.fset, self.fdel, self.__doc__)

Why was it changed?

Since the link is down I don't know the exact reason, however I can speculate based on the added test-cases in that commit:

import unittest

class PropertyBase(Exception):
    pass

class PropertyGet(PropertyBase):
    pass

class PropertySet(PropertyBase):
    pass

class PropertyDel(PropertyBase):
    pass

class BaseClass(object):
    def __init__(self):
        self._spam = 5

    @property
    def spam(self):
        """BaseClass.getter"""
        return self._spam

    @spam.setter
    def spam(self, value):
        self._spam = value

    @spam.deleter
    def spam(self):
        del self._spam

class SubClass(BaseClass):

    @BaseClass.spam.getter
    def spam(self):
        """SubClass.getter"""
        raise PropertyGet(self._spam)

    @spam.setter
    def spam(self, value):
        raise PropertySet(self._spam)

    @spam.deleter
    def spam(self):
        raise PropertyDel(self._spam)

class PropertyTests(unittest.TestCase):
    def test_property_decorator_baseclass(self):
        # see #1620
        base = BaseClass()
        self.assertEqual(base.spam, 5)
        self.assertEqual(base._spam, 5)
        base.spam = 10
        self.assertEqual(base.spam, 10)
        self.assertEqual(base._spam, 10)
        delattr(base, "spam")
        self.assert_(not hasattr(base, "spam"))
        self.assert_(not hasattr(base, "_spam"))
        base.spam = 20
        self.assertEqual(base.spam, 20)
        self.assertEqual(base._spam, 20)
        self.assertEqual(base.__class__.spam.__doc__, "BaseClass.getter")

    def test_property_decorator_subclass(self):
        # see #1620
        sub = SubClass()
        self.assertRaises(PropertyGet, getattr, sub, "spam")
        self.assertRaises(PropertySet, setattr, sub, "spam", None)
        self.assertRaises(PropertyDel, delattr, sub, "spam")
        self.assertEqual(sub.__class__.spam.__doc__, "SubClass.getter")

That's similar to the examples the other answers already provided. The problem is that you want to be able to change the behavior in a subclass without affecting the parent class:

>>> b = BaseClass()
>>> b.spam
5

However with your property it would result in this:

>>> b = BaseClass()
>>> b.spam
---------------------------------------------------------------------------
PropertyGet                               Traceback (most recent call last)
PropertyGet: 5

That happens because BaseClass.spam.getter (which is used in SubClass) actually modifies and returns the BaseClass.spam property!

So yes, it had been changed (very likely) because it allows to modify the behavior of a property in a subclass without changing the behavior on the parent class.

Another reason (?)

Note that there is an additional reason, which is a bit silly but actually worth mentioning (in my opinion):

Let's recap shortly: A decorator is just syntactic sugar for an assignment, so:

@decorator
def decoratee():
    pass

is equivalent to:

def func():
    pass

decoratee = decorator(func)
del func

The important point here is that the result of the decorator is assigned to the name of the decorated function. So while you generally use the same "function name" for the getter/setter/deleter - you don't have to!

For example:

class Fun(object):
    @property
    def a(self):
        return self._a

    @a.setter
    def b(self, value):
        self._a = value

>>> o = Fun()
>>> o.b = 100
>>> o.a
100
>>> o.b
100
>>> o.a = 100
AttributeError: can't set attribute

In this example you use the descriptor for a to create another descriptor for b that behaves like a except that it got a setter.

It's a rather weird example and probably not used very often (or at all). But even if it's rather odd and (to me) not very good style - it should illustrate that just because you use property_name.setter (or getter/deleter) that it has to be bound to property_name. It could be bound to any name! And I wouldn't expect it to propagate back to the original property (although I'm not really sure what I would expect here).

Summary

• CPython actually used the "modify and return self" approach in the getter, setter and deleter once.
• It had been changed because of a bug report.
• It behaved "buggy" when used with a subclass that overwrote a property of the parent class.
• More generally: Decorators cannot influence to what name they will be bound so the assumption that it's always valid to return self in a decorator might be questionable (for a general-purpose decorator).