Why does the class definition's metaclass keyword argument accept a callable?

Question

Background

The Python 3 documentation clearly describes how the metaclass of a class is determined:

• if no bases and no explicit metaclas

Answer 1

Concerning question 1, I think the "metaclass" of a class cls should be understood as type(cls). That way of understanding is compatible with Python's error message in the following example:

>>> class Meta1(type): pass
... 
>>> class Meta2(type): pass
... 
>>> def metafunc(name, bases, methods):
...     if methods.get('version') == 1:
...         return Meta1(name, bases, methods)
...     return Meta2(name, bases, methods)
... 
>>> class C1:
...     __metaclass__ = metafunc
...     version = 1
... 
>>> class C2:
...     __metaclass__ = metafunc
...     version = 2
... 
>>> type(C1)
<class '__main__.Meta1'>
>>> type(C2)
<class '__main__.Meta2'>
>>> class C3(C1,C2): pass
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Error when calling the metaclass bases
    metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

I.e., according to the error message, the metaclass of a class is a class, even though the callable used to construct the class can be just anything.

Concerning the second question, indeed with a subclass of type used as a metaclass, you can do the same as with any other callable. In particular, it is possible that it yields something that is not its instance:

>>> class Mockup(type):
...     def __new__(cls, name, bases, methods):
...         return Meta1(name, bases, methods)
... 
>>> class Foo:
...     __metaclass__ = Mockup
... 
>>> type(Foo)
<class '__main__.Meta1'>
>>> isinstance(Foo, Mockup)
False
>>> Foo.__metaclass__
<class '__main__.Mockup'>

As to why Python gives the freedom of using any callable: The previous example shows that it is actually irrelevant whether the callable is a type or not.

BTW, here is a fun example: It is possible to code metaclasses that, themselves, have a metaclass different from type---let's call it a metametaclass. The metametaclass implements what happens when a metaclass is called. In that way, it is possible to create a class with two bases whose metaclasses are not subclass of each other (compare with Python's error message in the example above!). Indeed, only the metaclass of the resulting class is subclass of the metaclass of the bases, and this metaclass is created on the fly:

>>> class MetaMeta(type):
...     def __call__(mcls, name, bases, methods):
...         metabases = set(type(X) for X in bases)
...         metabases.add(mcls)
...         if len(metabases) > 1:
...             mcls = type(''.join([X.__name__ for X in metabases]), tuple(metabases), {})
...         return mcls.__new__(mcls, name, bases, methods)
... 
>>> class Meta1(type):
...     __metaclass__ = MetaMeta
... 
>>> class Meta2(type):
...     __metaclass__ = MetaMeta
... 
>>> class C1:
...     __metaclass__ = Meta1
... 
>>> class C2:
...     __metaclass__ = Meta2
... 
>>> type(C1)
<class '__main__.Meta1'>
>>> type(C2)
<class '__main__.Meta2'>
>>> class C3(C1,C2): pass
... 
>>> type(C3)
<class '__main__.Meta1Meta2'>

What is less fun: The preceding example won't work in Python 3. If I understand correctly, Python 2 creates the class and checks whether its metaclass is a subclass of all its bases, whereas Python 3 first checks whether there is one base whose metaclass is superclass of the metaclasses of all other bases, and only then creates the new class. That's a regression, from my point of view. But that shall be the topic of a new question that I am about to post...

Edit: The new question is here

Answer 2

Regarding your first question the metaclass should be MyMetaclass (which it's so):

In [7]: print(type(MyClass), type(MyDerived))
<class '__main__.MyMetaclass'> <class '__main__.MyMetaclass'>

The reason is that if the metaclass is not an instance of type python calls the methaclass by passing these arguments to it name, bases, ns, **kwds (see new_class) and since you are returning your real metaclass in that function it gets the correct type for metaclass.

And about the second question:

What is the purpose of accepting an arbitrary callable?

There is no special purpose, it's actually the nature of metaclasses which is because that making an instance from a class always calls the metaclass by calling it's __call__ method:

Metaclass.__call__()

Which means that you can pass any callable as your metaclass. So for example if you test it with a nested function the result will still be the same:

In [21]: def metaclass_callable(name, bases, namespace):
             def inner():
                 return MyMetaclass(name, bases, namespace)
             return inner()
   ....: 

In [22]: class MyClass(metaclass=metaclass_callable):
             pass
   ....: 

In [23]: print(type(MyClass), type(MyDerived))
<class '__main__.MyMetaclass'> <class '__main__.MyMetaclass'>

---

For more info here is how Python crates a class:

It calls the new_class function which it calls prepare_class inside itself, then as you can see inside the prepare_class python calls the __prepare__ method of the appropriate metaclass, beside of finding the proper meta (using _calculate_meta function ) and creating the appropriate namespace for the class.

So all in one here is the hierarchy of executing a metacalss's methods:

1. __prepare__ ¹
2. __call__
3. __new__
4. __init__

And here is the source code:

# Provide a PEP 3115 compliant mechanism for class creation
def new_class(name, bases=(), kwds=None, exec_body=None):
    """Create a class object dynamically using the appropriate metaclass."""
    meta, ns, kwds = prepare_class(name, bases, kwds)
    if exec_body is not None:
        exec_body(ns)
    return meta(name, bases, ns, **kwds)

def prepare_class(name, bases=(), kwds=None):
    """Call the __prepare__ method of the appropriate metaclass.

    Returns (metaclass, namespace, kwds) as a 3-tuple

    *metaclass* is the appropriate metaclass
    *namespace* is the prepared class namespace
    *kwds* is an updated copy of the passed in kwds argument with any
    'metaclass' entry removed. If no kwds argument is passed in, this will
    be an empty dict.
    """
    if kwds is None:
        kwds = {}
    else:
        kwds = dict(kwds) # Don't alter the provided mapping
    if 'metaclass' in kwds:
        meta = kwds.pop('metaclass')
    else:
        if bases:
            meta = type(bases[0])
        else:
            meta = type
    if isinstance(meta, type):
        # when meta is a type, we first determine the most-derived metaclass
        # instead of invoking the initial candidate directly
        meta = _calculate_meta(meta, bases)
    if hasattr(meta, '__prepare__'):
        ns = meta.__prepare__(name, bases, **kwds)
    else:
        ns = {}
    return meta, ns, kwds


def _calculate_meta(meta, bases):
    """Calculate the most derived metaclass."""
    winner = meta
    for base in bases:
        base_meta = type(base)
        if issubclass(winner, base_meta):
            continue
        if issubclass(base_meta, winner):
            winner = base_meta
            continue
        # else:
        raise TypeError("metaclass conflict: "
                        "the metaclass of a derived class "
                        "must be a (non-strict) subclass "
                        "of the metaclasses of all its bases")
    return winner

---

_{1. Note that it get called implicitly inside the new_class function and before the return.}

Answer 3

Well, the type is of course MyMetaClass. metaclass_callable is initially 'selected' as the metaclass since it's been specified in the metaclass kwarg and as such, it's __call__ (a simple function call) is going to be performed.

It just so happens that calling it will print and then invoke MyMetaClass.__call__ (which calls type.__call__ since __call__ hasn't been overridden for MyMetaClass). There the assignment of cls.__class__ is made to MyMetaClass.

metaclass_callable is called once and then appears to be unrecoverable

Yes, it is only initially invoked and then hands control over to MyMetaClass. I'm not aware of any class attribute that keeps that information around.

derived classes do not use (as far as I can tell) metaclass_callable in any way.

Nope, if no metaclass is explicitly defined, the best match for the metaclasses of bases (here MyClass) will be used (resulting in MyMetaClass).

---

As for question 2, pretty sure everything you can do with a callable is also possible by using an instance of type with __call__ overridden accordingly. As to why, you might not want to go full blown class-creation if you simply want to make minor changes when actually creating a class.