Why does += behave unexpectedly on lists?

Question

The += operator in python seems to be operating unexpectedly on lists. Can anyone tell me what is going on here?

class foo:  
     bar = []
            


        

Answer 1

The general answer is that += tries to call the __iadd__ special method, and if that isn't available it tries to use __add__ instead. So the issue is with the difference between these special methods.

The __iadd__ special method is for an in-place addition, that is it mutates the object that it acts on. The __add__ special method returns a new object and is also used for the standard + operator.

So when the += operator is used on an object which has an __iadd__ defined the object is modified in place. Otherwise it will instead try to use the plain __add__ and return a new object.

That is why for mutable types like lists += changes the object's value, whereas for immutable types like tuples, strings and integers a new object is returned instead (a += b becomes equivalent to a = a + b).

For types that support both __iadd__ and __add__ you therefore have to be careful which one you use. a += b will call __iadd__ and mutate a, whereas a = a + b will create a new object and assign it to a. They are not the same operation!

>>> a1 = a2 = [1, 2]
>>> b1 = b2 = [1, 2]
>>> a1 += [3]          # Uses __iadd__, modifies a1 in-place
>>> b1 = b1 + [3]      # Uses __add__, creates new list, assigns it to b1
>>> a2
[1, 2, 3]              # a1 and a2 are still the same list
>>> b2
[1, 2]                 # whereas only b1 was changed

For immutable types (where you don't have an __iadd__) a += b and a = a + b are equivalent. This is what lets you use += on immutable types, which might seem a strange design decision until you consider that otherwise you couldn't use += on immutable types like numbers!

Answer 2

For the general case, see Scott Griffith's answer. When dealing with lists like you are, though, the += operator is a shorthand for someListObject.extend(iterableObject). See the documentation of extend().

The extend function will append all elements of the parameter to the list.

When doing foo += something you're modifying the list foo in place, thus you don't change the reference that the name foo points to, but you're changing the list object directly. With foo = foo + something, you're actually creating a new list.

This example code will explain it:

>>> l = []
>>> id(l)
13043192
>>> l += [3]
>>> id(l)
13043192
>>> l = l + [3]
>>> id(l)
13059216

Note how the reference changes when you reassign the new list to l.

As bar is a class variable instead of an instance variable, modifying in place will affect all instances of that class. But when redefining self.bar, the instance will have a separate instance variable self.bar without affecting the other class instances.