What are some (concrete) use-cases for metaclasses?

Question

I have a friend who likes to use metaclasses, and regularly offers them as a solution.

I am of the mind that you almost never need to use metaclasses. Why? because I

Answer 1

Metaclasses can be handy for construction of Domain Specific Languages in Python. Concrete examples are Django, SQLObject 's declarative syntax of database schemata.

A basic example from A Conservative Metaclass by Ian Bicking:

The metaclasses I've used have been primarily to support a sort of declarative style of programming. For instance, consider a validation schema:

class Registration(schema.Schema):
    first_name = validators.String(notEmpty=True)
    last_name = validators.String(notEmpty=True)
    mi = validators.MaxLength(1)
    class Numbers(foreach.ForEach):
        class Number(schema.Schema):
            type = validators.OneOf(['home', 'work'])
            phone_number = validators.PhoneNumber()

Some other techniques: Ingredients for Building a DSL in Python (pdf).

Edit (by Ali): An example of doing this using collections and instances is what I would prefer. The important fact is the instances, which give you more power, and eliminate reason to use metaclasses. Further worth noting that your example uses a mixture of classes and instances, which is surely an indication that you can't just do it all with metaclasses. And creates a truly non-uniform way of doing it.

number_validator = [
    v.OneOf('type', ['home', 'work']),
    v.PhoneNumber('phone_number'),
]

validators = [
    v.String('first_name', notEmpty=True),
    v.String('last_name', notEmpty=True),
    v.MaxLength('mi', 1),
    v.ForEach([number_validator,])
]

It's not perfect, but already there is almost zero magic, no need for metaclasses, and improved uniformity.

Answer 2

the answer from @Dan Gittik is cool

the examples at the end could clarify many things,I changed it to python 3 and give some explanation:

class MetaMetaclass(type):
    def __new__(meta, name, bases, attrs):
        def __new__(meta, name, bases, attrs):
            cls = type.__new__(meta, name, bases, attrs)
            cls._label = 'Made in %s' % meta.__name__
            return cls

        attrs['__new__'] = __new__
        return type.__new__(meta, name, bases, attrs)

#China is metaclass and it's __new__ method would be changed by MetaMetaclass(metaclass)
class China(MetaMetaclass, metaclass=MetaMetaclass):
    __metaclass__ = MetaMetaclass

#Taiwan is metaclass and it's __new__ method would be changed by MetaMetaclass(metaclass)
class Taiwan(MetaMetaclass, metaclass=MetaMetaclass):
    __metaclass__ = MetaMetaclass

#A is a normal class and it's __new__ method would be changed by China(metaclass)
class A(metaclass=China):
    __metaclass__ = China

#B is a normal class and it's __new__ method would be changed by Taiwan(metaclass)
class B(metaclass=Taiwan):
    __metaclass__ = Taiwan


print(A._label)  # Made in China
print(B._label)  # Made in Taiwan

• everything is object,so class is object
• class object is created by metaclass
• all class inheritted from type is metaclass
• metaclass could control class creating
• metaclass could control metaclass creating too(so it could loop for ever)
• this's metaprograming...you could control the type system at running time
• again,everything is object,this's a uniform system,type create type,and type create instance

Answer 3

I have a class that handles non-interactive plotting, as a frontend to Matplotlib. However, on occasion one wants to do interactive plotting. With only a couple functions I found that I was able to increment the figure count, call draw manually, etc, but I needed to do these before and after every plotting call. So to create both an interactive plotting wrapper and an offscreen plotting wrapper, I found it was more efficient to do this via metaclasses, wrapping the appropriate methods, than to do something like:

class PlottingInteractive:
    add_slice = wrap_pylab_newplot(add_slice)

This method doesn't keep up with API changes and so on, but one that iterates over the class attributes in __init__ before re-setting the class attributes is more efficient and keeps things up to date:

class _Interactify(type):
    def __init__(cls, name, bases, d):
        super(_Interactify, cls).__init__(name, bases, d)
        for base in bases:
            for attrname in dir(base):
                if attrname in d: continue # If overridden, don't reset
                attr = getattr(cls, attrname)
                if type(attr) == types.MethodType:
                    if attrname.startswith("add_"):
                        setattr(cls, attrname, wrap_pylab_newplot(attr))
                    elif attrname.startswith("set_"):
                        setattr(cls, attrname, wrap_pylab_show(attr))

Of course, there might be better ways to do this, but I've found this to be effective. Of course, this could also be done in __new__ or __init__, but this was the solution I found the most straightforward.

Answer 4

The only time I used metaclasses in Python was when writing a wrapper for the Flickr API.

My goal was to scrape flickr's api site and dynamically generate a complete class hierarchy to allow API access using Python objects:

# Both the photo type and the flickr.photos.search API method 
# are generated at "run-time"
for photo in flickr.photos.search(text=balloons):
    print photo.description

So in that example, because I generated the entire Python Flickr API from the website, I really don't know the class definitions at runtime. Being able to dynamically generate types was very useful.

Answer 5

A reasonable pattern of metaclass use is doing something once when a class is defined rather than repeatedly whenever the same class is instantiated.

When multiple classes share the same special behaviour, repeating __metaclass__=X is obviously better than repeating the special purpose code and/or introducing ad-hoc shared superclasses.

But even with only one special class and no foreseeable extension, __new__ and __init__ of a metaclass are a cleaner way to initialize class variables or other global data than intermixing special-purpose code and normal def and class statements in the class definition body.

Answer 6

I recently had to use a metaclass to help declaratively define an SQLAlchemy model around a database table populated with U.S. Census data from http://census.ire.org/data/bulkdata.html

IRE provides database shells for the census data tables, which create integer columns following a naming convention from the Census Bureau of p012015, p012016, p012017, etc.

I wanted to a) be able to access these columns using a model_instance.p012017 syntax, b) be fairly explicit about what I was doing and c) not have to explicitly define dozens of fields on the model, so I subclassed SQLAlchemy's DeclarativeMeta to iterate through a range of the columns and automatically create model fields corresponding to the columns:

from sqlalchemy.ext.declarative.api import DeclarativeMeta

class CensusTableMeta(DeclarativeMeta):
    def __init__(cls, classname, bases, dict_):
        table = 'p012'
        for i in range(1, 49):
            fname = "%s%03d" % (table, i)
            dict_[fname] = Column(Integer)
            setattr(cls, fname, dict_[fname])

        super(CensusTableMeta, cls).__init__(classname, bases, dict_)

I could then use this metaclass for my model definition and access the automatically enumerated fields on the model:

CensusTableBase = declarative_base(metaclass=CensusTableMeta)

class P12Tract(CensusTableBase):
    __tablename__ = 'ire_p12'

    geoid = Column(String(12), primary_key=True)

    @property
    def male_under_5(self):
        return self.p012003

    ...