super() and changing the signature of cooperative methods

Question

in a multiple inheritance setting such as laid out in, how can I use super() and also handle the case when the signature of the function changes between classes in

Answer 1

James Knight's article super() considered harmful suggests a solution by always accepting *args and **kwargs in all cooperating functions. however this solution does not work for two reasons:

1. object.__init__ does not accept arguments this is a breaking change introduced python 2.6 / 3.x TypeError: object.__init__() takes no parameters

2. using *args is actually counter productive

Solution TL;DR

1. super() usage has to be consistent: In a class hierarchy, super should be used everywhere or nowhere. is part of the contract of the class. if one classes uses super() all the classes MUST also use super() in the same way, or otherwise we might call certain functions in the hierarchy zero times, or more than once

2. to correctly support __init__ functions with any parameters, the top-level classes in your hierarchy must inherit from a custom class like SuperObject:

   class SuperObject:        
       def __init__(self, **kwargs):
           mro = type(self).__mro__
           assert mro[-1] is object
           if mro[-2] is not SuperObject:
               raise TypeError(
                   'all top-level classes in this hierarchy must inherit from SuperObject',
                   'the last class in the MRO should be SuperObject',
                   f'mro={[cls.__name__ for cls in mro]}'
               )
   
           # super().__init__ is guaranteed to be object.__init__        
           init = super().__init__
           init()
   

3. if overridden functions in the class hierarchy can take differing arguments, always pass all arguments you received on to the super function as keyword arguments, and, always accept **kwargs.

Here's a rewritten example

class A(SuperObject):
    def __init__(self, **kwargs):
        print("A")
        super(A, self).__init__(**kwargs)

class B(SuperObject):
    def __init__(self, **kwargs):
        print("B")
        super(B, self).__init__(**kwargs)

class C(A):
    def __init__(self, age, **kwargs):
        print("C",f"age={age}")
        super(C, self).__init__(age=age, **kwargs)

class D(B):
    def __init__(self, name, **kwargs):
        print("D", f"name={name}")
        super(D, self).__init__(name=name, **kwargs)

class E(C,D):
    def __init__(self, name, age, *args, **kwargs):
        print( "E", f"name={name}", f"age={age}")
        super(E, self).__init__(name=name, age=age, *args, **kwargs)

e = E(name='python', age=28)

output:

E name=python age=28
C age=28
A
D name=python
B
SuperObject

Discussion

lets look at both problems in more detail

object.__init__ does not accept arguments

consider the original solution given by James Knight:

the general rule is: always pass all arguments you received on to the super function, and, if classes can take differing arguments, always accept *args and **kwargs.

    class A:
        def __init__(self, *args, **kwargs):
            print("A")
            super().__init__(*args, **kwargs)

    class B(object):
        def __init__(self, *args, **kwargs):
            print("B")
            super().__init__(*args, **kwargs)

    class C(A):
        def __init__(self, arg, *args, **kwargs):
            print("C","arg=",arg)
            super().__init__(arg, *args, **kwargs)

    class D(B):
        def __init__(self, arg, *args, **kwargs):
            print("D", "arg=",arg)
            super().__init__(arg, *args, **kwargs)

    class E(C,D):
        def __init__(self, arg, *args, **kwargs):
            print( "E", "arg=",arg)
            super().__init__(arg, *args, **kwargs)

    print( "MRO:", [x.__name__ for x in E.__mro__])
    E(10)

a breaking change in python 2.6 and 3.x has changed object.__init__ signature so that it no longer accepts arbitrary arguments

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
 in 
     25 
     26 print( "MRO:", [x.__name__ for x in E.__mro__])
---> 27 E(10)

...

 in __init__(self, *args, **kwargs)
      7     def __init__(self, *args, **kwargs):
      8         print("B")
----> 9         super(B, self).__init__(*args, **kwargs)
     10 
     11 class C(A):

TypeError: object.__init__() takes exactly one argument (the instance to initialize)

The correct way to handle this conundrum is for the top level classes in a hierarchy to inherit from a custom class like SuperObject:

class SuperObject:        
    def __init__(self, *args, **kwargs):
        mro = type(self).__mro__
        assert mro[-1] is object
        if mro[-2] is not SuperObject:
            raise TypeError(
                'all top-level classes in this hierarchy must inherit from SuperObject',
                'the last class in the MRO should be SuperObject',
                f'mro={[cls.__name__ for cls in mro]}'
            )

        # super().__init__ is guaranteed to be object.__init__        
        init = super().__init__
        init()

and thus rewriting the example as follows should work

    class A(SuperObject):
        def __init__(self, *args, **kwargs):
            print("A")
            super(A, self).__init__(*args, **kwargs)

    class B(SuperObject):
        def __init__(self, *args, **kwargs):
            print("B")
            super(B, self).__init__(*args, **kwargs)

    class C(A):
        def __init__(self, arg, *args, **kwargs):
            print("C","arg=",arg)
            super(C, self).__init__(arg, *args, **kwargs)

    class D(B):
        def __init__(self, arg, *args, **kwargs):
            print("D", "arg=",arg)
            super(D, self).__init__(arg, *args, **kwargs)

    class E(C,D):
        def __init__(self, arg, *args, **kwargs):
            print( "E", "arg=",arg)
            super(E, self).__init__(arg, *args, **kwargs)

    print( "MRO:", [x.__name__ for x in E.__mro__])
    E(10)

output:

MRO: ['E', 'C', 'A', 'D', 'B', 'SuperObject', 'object']
E arg= 10
C arg= 10
A
D arg= 10
B
SuperObject

using *args is counter productive

lets make the example a bit more complicated, with two different parameters: name and age

class A(SuperObject):
    def __init__(self, *args, **kwargs):
        print("A")
        super(A, self).__init__(*args, **kwargs)

class B(SuperObject):
    def __init__(self, *args, **kwargs):
        print("B")
        super(B, self).__init__(*args, **kwargs)

class C(A):
    def __init__(self, age, *args, **kwargs):
        print("C",f"age={age}")
        super(C, self).__init__(age, *args, **kwargs)

class D(B):
    def __init__(self, name, *args, **kwargs):
        print("D", f"name={name}")
        super(D, self).__init__(name, *args, **kwargs)

class E(C,D):
    def __init__(self, name, age, *args, **kwargs):
        print( "E", f"name={name}", f"age={age}")
        super(E, self).__init__(name, age, *args, **kwargs)

E('python', 28)

output:

E name=python age=28
C age=python
A
D name=python
B
SuperObject

as you can see from the line C age=python the positional arguments got confused and we're passing the wrong thing along.

my suggested solution is to be more strict and avoid an *args argument altogether. instead:

if classes can take differing arguments, always pass all arguments you received on to the super function as keyword arguments, and, always accept **kwargs.

here's a solution based on this stricter rule. first remove *args from SuperObject

class SuperObject:        
    def __init__(self, **kwargs):
        print('SuperObject')
        mro = type(self).__mro__
        assert mro[-1] is object
        if mro[-2] is not SuperObject:
            raise TypeError(
                'all top-level classes in this hierarchy must inherit from SuperObject',
                'the last class in the MRO should be SuperObject',
                f'mro={[cls.__name__ for cls in mro]}'
            )

        # super().__init__ is guaranteed to be object.__init__        
        init = super().__init__
        init()

and now remove *args from the rest of the classes, and pass arguments by name only

class A(SuperObject):
    def __init__(self, **kwargs):
        print("A")
        super(A, self).__init__(**kwargs)

class B(SuperObject):
    def __init__(self, **kwargs):
        print("B")
        super(B, self).__init__(**kwargs)

class C(A):
    def __init__(self, age, **kwargs):
        print("C",f"age={age}")
        super(C, self).__init__(age=age, **kwargs)

class D(B):
    def __init__(self, name, **kwargs):
        print("D", f"name={name}")
        super(D, self).__init__(name=name, **kwargs)

class E(C,D):
    def __init__(self, name, age, *args, **kwargs):
        print( "E", f"name={name}", f"age={age}")
        super(E, self).__init__(name=name, age=age, *args, **kwargs)

E(name='python', age=28)

output:

E name=python age=28
C age=28
A
D name=python
B
SuperObject

which is correct