Cannot inherit from multiple classes defining __slots__?

Question

A certain situation in Python recently alarmed me, and its reason is still not completely clear after a little research. The following class definitions appear to work flawl

Answer 1

Have you seen this alternative? https://stackoverflow.com/a/53063670/1400467

There's a "tricky" workaround by using metaclasses and a fake _slots_ attribute. This works in Python 3.6 and hope in Python 3.X so.

Answer 2

By forcing a constraint that no class defines __slots__, a special object class could be constructed with the characteristics desired for all child classes. The class is registered as an alias for regular objects.

class _object: __slots__ = '_MetaSafe__exec', '__dict__'

class MetaSafe(type):

    __REGISTRY = {object: _object}

    @classmethod
    def clone(cls, old):
        return cls(old.__name__, old.__bases__, dict(old.__dict__), old)

    def __new__(cls, name, bases, classdict, old=None):
        # Check on a few classdict keys.
        assert '__new__' not in classdict, '__new__ must not be defined!'
        assert '__slots__' not in classdict, '__slots__ must not be defined!'
        assert '__module__' in classdict, '__module__ must be defined!'
        # Validate all the parent classes.
        valid = []
        for base in bases:
            if base in cls.__REGISTRY:
                valid.append(cls.__REGISTRY[base])
            elif base in cls.__REGISTRY.values():
                valid.append(base)
            else:
                valid.append(cls.clone(base))
        # Wrap callables without thread mark.
        for key, value in classdict.items():
            if callable(value):
                classdict[key] = cls.__wrap(value)
        # Fix classdict and create new class.
        classdict.update({'__new__': cls.__new, '__slots__': (), '__module__':
                          '{}.{}'.format(__name__, classdict['__module__'])})
        cls.__REGISTRY[old] = new = \
            super().__new__(cls, name, tuple(valid), classdict)
        return new

    def __init__(self, name, bases, classdict, old=None):
        return super().__init__(name, bases, classdict)

    @staticmethod
    def __wrap(func):
        @functools.wraps(func)
        def safe(self, *args, **kwargs):
            return self.__exec(func, self, *args, **kwargs)
        return safe

    @classmethod
    def __new(meta, cls, *args, **kwargs):
        self = object.__new__(cls, *args, **kwargs)
        if 'master' in kwargs:
            self.__exec = kwargs['master'].__exec
        else:
            array = tuple(meta.__REGISTRY.values())
            for value in args:
                if isinstance(value, array):
                    self.__exec = value.__exec
                    break
            else:
                self.__exec = Affinity()
        return self

This code can be used as a building block to make tkinter thread-safe by cloning its classes. The Affinity class automatically ensures that code is executed on a single thread, preventing GUI errors.

Answer 3

Cannot inherit from multiple classes defining __slots__?

Close.

You cannot inherit from multiple classes defining nonempty __slots__ when there is a layout conflict.

Slots have an ordered layout, and the descriptors that get created in the class rely on those positions, therefore they must not have a layout conflict under multiple inheritance.

Your simplest approach fails because each a and b are considered different slots, and the layout algorithm does not check whether they are semantically the same:

class B: __slots__ = 'a', 'b' # creates descriptors in B for a, b
class C: __slots__ = 'a', 'b' # creates new, different descriptors in C
class D(B, C): __slots__ = () # B.a or C.a comes first?

Your first example works because the multiple inheritance gets only A's slots, thus all cases are using A's descriptors and positions/layout. For example, the following would be allowed:

class A: __slots__ = 'a', 'b' # shared parent, ok

class B(A): __slots__ = () # B or C must be empty

class C(A): __slots__ = 'c', # Since C is nonempty, B must be empty to inherit from both

class D(B, C): __slots__ = 'd', 'e'

Instantiating D, and using those slots:

d = D()
d.a = d.b = d.c = d.d = d.e = 'foo'

And we cannot dynamically create variables:

>>> d.f = 'foo'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'D' object has no attribute 'f'

The above is one approach to solving your problematic code, but it could require a bit of rewriting - if you decide B needs another slot, you've got to refactor B's functionality into an abstraction to get code reuse for D (which is fine, but potentially confusing).

It's a best practice to use abstractions, and another solution would be to do this, where the abstract classes and/or mixins contain the functionality for your concrete classes:

class AbstractB: __slots__ = ()

class B(AbstractB): __slots__ = 'a', 'b'

class AbstractC: __slots__ = ()

class C(AbstractC): __slots__ = 'a', 'b'

class Mixin: __slots__ = ()

class D(AbstractB, AbstractC, Mixin): __slots__ = 'a', 'b'

Your first example is quite workable because it avoids a layout conflict, this just reimagines a solution using abstractions instead of concretions.

Final Questions:

(1) Is this a bug in Python, considering the slot names?

No, in spite of lots of confusion on the matter, it's somewhat documented and the errors try to make this behavior clear.

(2) What would justify such an answer?

Classes that define slots get descriptors that know where their data goes positionally. If layouts change, the descriptors would be wrong.

Could each subclass create its own layout and its own descriptors? I suppose it could, but that would require a bit of rewriting of how they work, and some political will to do it, and could conceivably break other users that are poking around in the C api and relying on the current behavior.

(3) What is the best workaround?

Define "best".

Fastest to write and possibly least complex?: just avoid layout conflicts like in your first example.

Best practices?: Use abstract inheritance trees, and define slots in your concretions. While there may be more classes with this approach, it may be arguably less complex for others and "future-you" to deal with.

Answer 4

I've faced this error and I really wanted to use slots for my custom database nodes. Here is the test suite I've made (Its in Python 3.x):

import logging

A = None, 'attr1', 'attr2', 'attr3', 'attr4'

class C12(object):
    __slots__ = (A[1], A[2])

class C1234(object):
    __slots__ = (A[1], A[2], A[3], A[4])

class C34(object):
    __slots__ = (A[3], A[4])

class C3byC12(C12):
    __slots__ = (A[3])

class CEmpty(object):
    __slots__ = ()

MSG_FRM = '\n\tc1: {}\n\tc2: {}\n\t__slots__: {}'
NOT_DEF = 'not defined'

def test(c1, c2, slots):
    logging.debug('*'*20 + ' new class test ' + '*'*20)
    msg = MSG_FRM.format(c1, c2, slots)
    try:
        if slots == NOT_DEF:
            class TestClass(c1, c2): pass
        else:        
            class TestClass(c1, c2):
                __slots__ = slots
    except TypeError:
        logging.exception('BOOM!!! ' + msg)
    else:
        logging.debug('No Boom! ' + msg)
        instance = TestClass()
        if '__dict__' in dir(instance):
            logging.warning('Instance has __dict__!')
        else:
            logging.debug('Instance __slots__:{}'.format(
                          instance.__slots__))
        logging.debug('Attributes in instance dir: {}'.format(
            ' '.join(['X' if (a in dir(instance)) else '_'
                     for a in A[1:]])))

if __name__ == '__main__':
    logging.basicConfig(level=logging.DEBUG)
    test(C12, C34, (A[2], A[4]))
    test(C12, C3byC12, (A[2],))
    test(C3byC12, C12, (A[4],))
    test(C1234, C34, (A[2], A[4]))
    test(C1234, CEmpty, (A[2], A[4]))
    test(C12, CEmpty, (A[2], A[4]))
    test(C12, CEmpty, (A[1], A[2]))
    test(C12, CEmpty, ())
    test(CEmpty, C1234, (A[2], A[4]))
    test(CEmpty, C12, (A[3],))
    test(C12, C34, NOT_DEF)
    test(C12, CEmpty, NOT_DEF)

Here are the results:

DEBUG:root:******************** new class test ********************
ERROR:root:BOOM!!!
        c1: <class '__main__.C12'>
        c2: <class '__main__.C34'>
        __slots__: ('attr2', 'attr4')
Traceback (most recent call last):
  File "boom.py", line 30, in test
    class TestClass(c1, c2):
TypeError: multiple bases have instance lay-out conflict
DEBUG:root:******************** new class test ********************
ERROR:root:BOOM!!!
        c1: <class '__main__.C12'>
        c2: <class '__main__.C3byC12'>
        __slots__: ('attr2',)
Traceback (most recent call last):
  File "boom.py", line 30, in test
    class TestClass(c1, c2):
TypeError: Cannot create a consistent method resolution
order (MRO) for bases C3byC12, C12
DEBUG:root:******************** new class test ********************
DEBUG:root:No Boom!
        c1: <class '__main__.C3byC12'>
        c2: <class '__main__.C12'>
        __slots__: ('attr4',)
DEBUG:root:Instance __slots__:('attr4',)
DEBUG:root:Attributes in instance dir: X X X X
DEBUG:root:******************** new class test ********************
ERROR:root:BOOM!!!
        c1: <class '__main__.C1234'>
        c2: <class '__main__.C34'>
        __slots__: ('attr2', 'attr4')
Traceback (most recent call last):
  File "boom.py", line 30, in test
    class TestClass(c1, c2):
TypeError: multiple bases have instance lay-out conflict
DEBUG:root:******************** new class test ********************
DEBUG:root:No Boom!
        c1: <class '__main__.C1234'>
        c2: <class '__main__.CEmpty'>
        __slots__: ('attr2', 'attr4')
DEBUG:root:Instance __slots__:('attr2', 'attr4')
DEBUG:root:Attributes in instance dir: X X X X
DEBUG:root:******************** new class test ********************
DEBUG:root:No Boom!
        c1: <class '__main__.C12'>
        c2: <class '__main__.CEmpty'>
        __slots__: ('attr2', 'attr4')
DEBUG:root:Instance __slots__:('attr2', 'attr4')
DEBUG:root:Attributes in instance dir: X X _ X
DEBUG:root:******************** new class test ********************
DEBUG:root:No Boom!
        c1: <class '__main__.C12'>
        c2: <class '__main__.CEmpty'>
        __slots__: ('attr1', 'attr2')
DEBUG:root:Instance __slots__:('attr1', 'attr2')
DEBUG:root:Attributes in instance dir: X X _ _
DEBUG:root:******************** new class test ********************
DEBUG:root:No Boom!
        c1: <class '__main__.C12'>
        c2: <class '__main__.CEmpty'>
        __slots__: ()
DEBUG:root:Instance __slots__:()
DEBUG:root:Attributes in instance dir: X X _ _
DEBUG:root:******************** new class test ********************
DEBUG:root:No Boom!
        c1: <class '__main__.CEmpty'>
        c2: <class '__main__.C1234'>
        __slots__: ('attr2', 'attr4')
DEBUG:root:Instance __slots__:('attr2', 'attr4')
DEBUG:root:Attributes in instance dir: X X X X
DEBUG:root:******************** new class test ********************
DEBUG:root:No Boom!
        c1: <class '__main__.CEmpty'>
        c2: <class '__main__.C12'>
        __slots__: ('attr3',)
DEBUG:root:Instance __slots__:('attr3',)
DEBUG:root:Attributes in instance dir: X X X _
DEBUG:root:******************** new class test ********************
ERROR:root:BOOM!!!
        c1: <class '__main__.C12'>
        c2: <class '__main__.C34'>
        __slots__: not defined
Traceback (most recent call last):
  File "boom.py", line 28, in test
    class TestClass(c1, c2): pass
TypeError: multiple bases have instance lay-out conflict
DEBUG:root:******************** new class test ********************
DEBUG:root:No Boom!
        c1: <class '__main__.C12'>
        c2: <class '__main__.CEmpty'>
        __slots__: not defined
WARNING:root:Instance has __dict__!
DEBUG:root:Attributes in instance dir: X X _ _

As you can see you have two options:

1. Either define __slots__ = () for all but one of the parent classes,
2. or make one of the parents to subclass of the another.

Note that you should define __slots__ in the new class too, otherwise it gets a __dict__.