What are some (concrete) use-cases for metaclasses?

Question

I have a friend who likes to use metaclasses, and regularly offers them as a solution.

I am of the mind that you almost never need to use metaclasses. Why? because I

Answer 1

The purpose of metaclasses isn't to replace the class/object distinction with metaclass/class - it's to change the behaviour of class definitions (and thus their instances) in some way. Effectively it's to alter the behaviour of the class statement in ways that may be more useful for your particular domain than the default. The things I have used them for are:

• Tracking subclasses, usually to register handlers. This is handy when using a plugin style setup, where you wish to register a handler for a particular thing simply by subclassing and setting up a few class attributes. eg. suppose you write a handler for various music formats, where each class implements appropriate methods (play / get tags etc) for its type. Adding a handler for a new type becomes:

  class Mp3File(MusicFile):
      extensions = ['.mp3']  # Register this type as a handler for mp3 files
      ...
      # Implementation of mp3 methods go here
  

  The metaclass then maintains a dictionary of {'.mp3' : MP3File, ... } etc, and constructs an object of the appropriate type when you request a handler through a factory function.

• Changing behaviour. You may want to attach a special meaning to certain attributes, resulting in altered behaviour when they are present. For example, you may want to look for methods with the name _get_foo and _set_foo and transparently convert them to properties. As a real-world example, here's a recipe I wrote to give more C-like struct definitions. The metaclass is used to convert the declared items into a struct format string, handling inheritance etc, and produce a class capable of dealing with it.

  For other real-world examples, take a look at various ORMs, like sqlalchemy's ORM or sqlobject. Again, the purpose is to interpret defintions (here SQL column definitions) with a particular meaning.

Answer 2

I had to use them once for a binary parser to make it easier to use. You define a message class with attributes of the fields present on the wire. They needed to be ordered in the way they were declared to construct the final wire format from it. You can do that with metaclasses, if you use an ordered namespace dict. In fact, its in the examples for Metaclasses:

https://docs.python.org/3/reference/datamodel.html#metaclass-example

But in general: Very carefully evaluate, if you really really need the added complexity of metaclasses.

Answer 3

Let's start with Tim Peter's classic quote:

Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don't (the people who actually need them know with certainty that they need them, and don't need an explanation about why). Tim Peters (c.l.p post 2002-12-22)

Having said that, I have (periodically) run across true uses of metaclasses. The one that comes to mind is in Django where all of your models inherit from models.Model. models.Model, in turn, does some serious magic to wrap your DB models with Django's ORM goodness. That magic happens by way of metaclasses. It creates all manner of exception classes, manager classes, etc. etc.

See django/db/models/base.py, class ModelBase() for the beginning of the story.

Answer 4

Metaclasses aren't replacing programming! They're just a trick which can automate or make more elegant some tasks. A good example of this is Pygments syntax highlighting library. It has a class called RegexLexer which lets the user define a set of lexing rules as regular expressions on a class. A metaclass is used to turn the definitions into a useful parser.

They're like salt; it's easy to use too much.

Answer 5

This is a minor use, but... one thing I've found metaclasses useful for is to invoke a function whenever a subclass is created. I codified this into a metaclass which looks for an __initsubclass__ attribute: whenever a subclass is created, all parent classes which define that method are invoked with __initsubclass__(cls, subcls). This allows creation of a parent class which then registers all subclasses with some global registry, runs invariant checks on subclasses whenever they are defined, perform late-binding operations, etc... all without have to manually call functions or to create custom metaclasses that perform each of these separate duties.

Mind you, I've slowly come to realize the implicit magicalness of this behavior is somewhat undesirable, since it's unexpected if looking at a class definition out of context... and so I've moved away from using that solution for anything serious besides initializing a __super attribute for each class and instance.

Answer 6

I was thinking the same thing just yesterday and completely agree. The complications in the code caused by attempts to make it more declarative generally make the codebase harder to maintain, harder to read and less pythonic in my opinion. It also normally requires a lot of copy.copy()ing (to maintain inheritance and to copy from class to instance) and means you have to look in many places to see whats going on (always looking from metaclass up) which goes against the python grain also. I have been picking through formencode and sqlalchemy code to see if such a declarative style was worth it and its clearly not. Such style should be left to descriptors (such as property and methods) and immutable data. Ruby has better support for such declarative styles and I am glad the core python language is not going down that route.

I can see their use for debugging, add a metaclass to all your base classes to get richer info. I also see their use only in (very) large projects to get rid of some boilerplate code (but at the loss of clarity). sqlalchemy for example does use them elsewhere, to add a particular custom method to all subclasses based on an attribute value in their class definition e.g a toy example

class test(baseclass_with_metaclass):
    method_maker_value = "hello"

could have a metaclass that generated a method in that class with special properties based on "hello" (say a method that added "hello" to the end of a string). It could be good for maintainability to make sure you did not have to write a method in every subclass you make instead all you have to define is method_maker_value.

The need for this is so rare though and only cuts down on a bit of typing that its not really worth considering unless you have a large enough codebase.