Python property descriptor design: why copy rather than mutate?

Question

I was looking at how Python implements the property descriptor internally. According to the docs property() is implemented in terms of the descriptor protocol, repr

Answer 1

TL;DR - return self permits child classes to change the behaviour of their parents. See MCVE of the failure below.

When you create property x in a parent class, that class has an attribute x with a particular setter, getter, and deleter. The first time you say @Parent.x.getter or the like in a child class, you are invoking a method on the parent's x member. If x.getter did not copy the property instance, calling it from the child class would change the parent's getter. That would prevent the parent class from operating the way it was designed to. (Thanks to Martijn Pieters (no surprise) here.)

And besides, the docs require it:

A property object has getter, setter, and deleter methods usable as decorators that create a copy of the property ...

An example, showing the internals:

class P:
    ## @property  --- inner workings shown below, marked "##"
    def x(self):
        return self.__x
    x = property(x)                             ## what @property does

    ## @x.setter
    def some_internal_name(self, x):
        self.__x = x
    x = x.setter(some_internal_name)            ## what @x.setter does

class C(P):
    ## @P.x.getter   # x is defined in parent P, so you have to specify P.x
    def another_internal_name(self):
        return 42

    # Remember, P.x is defined in the parent.  
    # If P.x.getter changes self, the parent's P.x changes.
    x = P.x.getter(another_internal_name)         ## what @P.x.getter does
    # Now an x exists in the child as well as in the parent. 

If getter mutated and returned self as you suggested, the child's x would be exactly the parent's x, and both would have been modified.

However, because the spec requires getter to return a copy, the child's x is a new copy with another_internal_name as fget, and the parent's x is untouched.

MCVE

It's a bit long, but shows the behaviour on Py 2.7.14.

class OopsProperty(object):
    "Shows what happens if getter()/setter()/deleter() don't copy"

    def __init__(self, fget=None, fset=None, fdel=None, doc=None):
        self.fget = fget
        self.fset = fset
        self.fdel = fdel
        if doc is None and fget is not None:
            doc = fget.__doc__
        self.__doc__ = doc

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        if self.fget is None:
            raise AttributeError("unreadable attribute")
        return self.fget(obj)

    def __set__(self, obj, value):
        if self.fset is None:
            raise AttributeError("can't set attribute")
        self.fset(obj, value)

    def __delete__(self, obj):
        if self.fdel is None:
            raise AttributeError("can't delete attribute")
        self.fdel(obj)

    ########## getter/setter/deleter modified as the OP suggested
    def getter(self, fget):
        self.fget = fget
        return self

    def setter(self, fset):
        self.fset = fset
        return self

    def deleter(self, fdel):
        self.fdel = fdel
        return self

class OopsParent(object):   # Uses OopsProperty() instead of property()
    def __init__(self):
        self.__x = 0

    @OopsProperty
    def x(self):
        print("OopsParent.x getter")
        return self.__x

    @x.setter
    def x(self, x):
        print("OopsParent.x setter")
        self.__x = x

class OopsChild(OopsParent):
    @OopsParent.x.getter                 # changes OopsParent.x!
    def x(self):
        print("OopsChild.x getter")
        return 42;

parent = OopsParent()
print("OopsParent x is",parent.x);

child = OopsChild()
print("OopsChild x is",child.x);

class Parent(object):   # Same thing, but using property()
    def __init__(self):
        self.__x = 0

    @property
    def x(self):
        print("Parent.x getter")
        return self.__x

    @x.setter
    def x(self, x):
        print("Parent.x setter")
        self.__x = x

class Child(Parent):
    @Parent.x.getter
    def x(self):
        print("Child.x getter")
        return 42;

parent = Parent()
print("Parent x is",parent.x);

child = Child()
print("Child x is",child.x);

And the run:

$ python foo.py
OopsChild.x getter              <-- Oops!  parent.x called the child's getter
('OopsParent x is', 42)         <-- Oops!
OopsChild.x getter
('OopsChild x is', 42)
Parent.x getter                 <-- Using property(), it's OK
('Parent x is', 0)              <-- What we expected from the parent class
Child.x getter
('Child x is', 42)

Answer 2

Let's start with a bit of history, because the original implementation had been equivalent to your alternative (equivalent because property is implemented in C in CPython so the getter, etc. are written in C not "plain Python").

However it was reported as issue (1620) on the Python bug tracker back in 2007:

As reported by Duncan Booth at http://permalink.gmane.org/gmane.comp.python.general/551183 the new @spam.getter syntax modifies the property in place but it should create a new one.

The patch is the first draft of a fix. I've to write unit tests to verify the patch. It copies the property and as a bonus grabs the __doc__ string from the getter if the doc string initially came from the getter as well.

Unfortunately the link doesn't go anywhere (I really don't know why it's called a "permalink" ...). It was classified as bug and changed to the current form (see this patch or the corresponding Github commit (but it's a combination of several patches)). In case you don't want to follow the link the change was:

 PyObject *
 property_getter(PyObject *self, PyObject *getter)
 {
-   Py_XDECREF(((propertyobject *)self)->prop_get);
-   if (getter == Py_None)
-       getter = NULL;
-   Py_XINCREF(getter);
-   ((propertyobject *)self)->prop_get = getter;
-   Py_INCREF(self);
-   return self;
+   return property_copy(self, getter, NULL, NULL, NULL);
 }

And similar for setter and deleter. If you don't know C the important lines are:

((propertyobject *)self)->prop_get = getter;

and

return self;

the rest is mostly "Python C API boilerplate". However these two lines are equivalent to your:

self.fget = fget
return self

And it was changed to:

return property_copy(self, getter, NULL, NULL, NULL);

which essentially does:

return type(self)(fget, self.fset, self.fdel, self.__doc__)

Why was it changed?

Since the link is down I don't know the exact reason, however I can speculate based on the added test-cases in that commit:

import unittest

class PropertyBase(Exception):
    pass

class PropertyGet(PropertyBase):
    pass

class PropertySet(PropertyBase):
    pass

class PropertyDel(PropertyBase):
    pass

class BaseClass(object):
    def __init__(self):
        self._spam = 5

    @property
    def spam(self):
        """BaseClass.getter"""
        return self._spam

    @spam.setter
    def spam(self, value):
        self._spam = value

    @spam.deleter
    def spam(self):
        del self._spam

class SubClass(BaseClass):

    @BaseClass.spam.getter
    def spam(self):
        """SubClass.getter"""
        raise PropertyGet(self._spam)

    @spam.setter
    def spam(self, value):
        raise PropertySet(self._spam)

    @spam.deleter
    def spam(self):
        raise PropertyDel(self._spam)

class PropertyTests(unittest.TestCase):
    def test_property_decorator_baseclass(self):
        # see #1620
        base = BaseClass()
        self.assertEqual(base.spam, 5)
        self.assertEqual(base._spam, 5)
        base.spam = 10
        self.assertEqual(base.spam, 10)
        self.assertEqual(base._spam, 10)
        delattr(base, "spam")
        self.assert_(not hasattr(base, "spam"))
        self.assert_(not hasattr(base, "_spam"))
        base.spam = 20
        self.assertEqual(base.spam, 20)
        self.assertEqual(base._spam, 20)
        self.assertEqual(base.__class__.spam.__doc__, "BaseClass.getter")

    def test_property_decorator_subclass(self):
        # see #1620
        sub = SubClass()
        self.assertRaises(PropertyGet, getattr, sub, "spam")
        self.assertRaises(PropertySet, setattr, sub, "spam", None)
        self.assertRaises(PropertyDel, delattr, sub, "spam")
        self.assertEqual(sub.__class__.spam.__doc__, "SubClass.getter")

That's similar to the examples the other answers already provided. The problem is that you want to be able to change the behavior in a subclass without affecting the parent class:

>>> b = BaseClass()
>>> b.spam
5

However with your property it would result in this:

>>> b = BaseClass()
>>> b.spam
---------------------------------------------------------------------------
PropertyGet                               Traceback (most recent call last)
PropertyGet: 5

That happens because BaseClass.spam.getter (which is used in SubClass) actually modifies and returns the BaseClass.spam property!

So yes, it had been changed (very likely) because it allows to modify the behavior of a property in a subclass without changing the behavior on the parent class.

Another reason (?)

Note that there is an additional reason, which is a bit silly but actually worth mentioning (in my opinion):

Let's recap shortly: A decorator is just syntactic sugar for an assignment, so:

@decorator
def decoratee():
    pass

is equivalent to:

def func():
    pass

decoratee = decorator(func)
del func

The important point here is that the result of the decorator is assigned to the name of the decorated function. So while you generally use the same "function name" for the getter/setter/deleter - you don't have to!

For example:

class Fun(object):
    @property
    def a(self):
        return self._a

    @a.setter
    def b(self, value):
        self._a = value

>>> o = Fun()
>>> o.b = 100
>>> o.a
100
>>> o.b
100
>>> o.a = 100
AttributeError: can't set attribute

In this example you use the descriptor for a to create another descriptor for b that behaves like a except that it got a setter.

It's a rather weird example and probably not used very often (or at all). But even if it's rather odd and (to me) not very good style - it should illustrate that just because you use property_name.setter (or getter/deleter) that it has to be bound to property_name. It could be bound to any name! And I wouldn't expect it to propagate back to the original property (although I'm not really sure what I would expect here).

Summary

• CPython actually used the "modify and return self" approach in the getter, setter and deleter once.
• It had been changed because of a bug report.
• It behaved "buggy" when used with a subclass that overwrote a property of the parent class.
• More generally: Decorators cannot influence to what name they will be bound so the assumption that it's always valid to return self in a decorator might be questionable (for a general-purpose decorator).

Answer 3

So you can use properties with inheritance?

Just an attempt at answering by giving an example:

class Base(object):
    def __init__(self):
        self._value = 0

    @property
    def value(self):
        return self._value

    @value.setter
    def value(self, val):
        self._value = val


class Child(Base):
    def __init__(self):
        super().__init__()
        self._double = 0

    @Base.value.setter
    def value(self, val):
        Base.value.fset(self, val)
        self._double = val * 2

If it was implemented the way you write it, then the Base.value.setter would also set the double, which is not wanted. We want a brand new setter, not to modify the base one.

EDIT: as pointed out by @wim, in this particular case, not only it would modify the base setter, but we would also end up with a recursion error. Indeed the child setter would call the base one, which would be modified to call itself with Base.value.fset in an endless recursion.