Howto get all methods of a python class with given decorator

Question

How to get all methods of a given class A that are decorated with the @decorator2?

class A():
    def method_a(self):
      pass

    @decorator1
    def method_b(self, b):
      pass

    @decorator2
    def method_c(self, t=5):
      pass

Answer 1

Method 1: Basic registering decorator

I already answered this question here: Calling functions by array index in Python =)

---

Method 2: Sourcecode parsing

If you do not have control over the class definition, which is one interpretation of what you'd like to suppose, this is impossible (without code-reading-reflection), since for example the decorator could be a no-op decorator (like in my linked example) that merely returns the function unmodified. (Nevertheless if you allow yourself to wrap/redefine the decorators, see Method 3: Converting decorators to be "self-aware", then you will find an elegant solution)

It is a terrible terrible hack, but you could use the inspect module to read the sourcecode itself, and parse it. This will not work in an interactive interpreter, because the inspect module will refuse to give sourcecode in interactive mode. However, below is a proof of concept.

#!/usr/bin/python3

import inspect

def deco(func):
    return func

def deco2():
    def wrapper(func):
        pass
    return wrapper

class Test(object):
    @deco
    def method(self):
        pass

    @deco2()
    def method2(self):
        pass

def methodsWithDecorator(cls, decoratorName):
    sourcelines = inspect.getsourcelines(cls)[0]
    for i,line in enumerate(sourcelines):
        line = line.strip()
        if line.split('(')[0].strip() == '@'+decoratorName: # leaving a bit out
            nextLine = sourcelines[i+1]
            name = nextLine.split('def')[1].split('(')[0].strip()
            yield(name)

It works!:

>>> print(list(  methodsWithDecorator(Test, 'deco')  ))
['method']

Note that one has to pay attention to parsing and the python syntax, e.g. @deco and @deco(... are valid results, but @deco2 should not be returned if we merely ask for 'deco'. We notice that according to the official python syntax at http://docs.python.org/reference/compound_stmts.html decorators are as follows:

decorator      ::=  "@" dotted_name ["(" [argument_list [","]] ")"] NEWLINE

We breathe a sigh of relief at not having to deal with cases like @(deco). But note that this still doesn't really help you if you have really really complicated decorators, such as @getDecorator(...), e.g.

def getDecorator():
    return deco

Thus, this best-that-you-can-do strategy of parsing code cannot detect cases like this. Though if you are using this method, what you're really after is what is written on top of the method in the definition, which in this case is getDecorator.

According to the spec, it is also valid to have @foo1.bar2.baz3(...) as a decorator. You can extend this method to work with that. You might also be able to extend this method to return a <function object ...> rather than the function's name, with lots of effort. This method however is hackish and terrible.

---

Method 3: Converting decorators to be "self-aware"

If you do not have control over the decorator definition (which is another interpretation of what you'd like), then all these issues go away because you have control over how the decorator is applied. Thus, you can modify the decorator by wrapping it, to create your own decorator, and use that to decorate your functions. Let me say that yet again: you can make a decorator that decorates the decorator you have no control over, "enlightening" it, which in our case makes it do what it was doing before but also append a .decorator metadata property to the callable it returns, allowing you to keep track of "was this function decorated or not? let's check function.decorator!". And then you can iterate over the methods of the class, and just check to see if the decorator has the appropriate .decorator property! =) As demonstrated here:

def makeRegisteringDecorator(foreignDecorator):
    """
        Returns a copy of foreignDecorator, which is identical in every
        way(*), except also appends a .decorator property to the callable it
        spits out.
    """
    def newDecorator(func):
        # Call to newDecorator(method)
        # Exactly like old decorator, but output keeps track of what decorated it
        R = foreignDecorator(func) # apply foreignDecorator, like call to foreignDecorator(method) would have done
        R.decorator = newDecorator # keep track of decorator
        #R.original = func         # might as well keep track of everything!
        return R

    newDecorator.__name__ = foreignDecorator.__name__
    newDecorator.__doc__ = foreignDecorator.__doc__
    # (*)We can be somewhat "hygienic", but newDecorator still isn't signature-preserving, i.e. you will not be able to get a runtime list of parameters. For that, you need hackish libraries...but in this case, the only argument is func, so it's not a big issue

    return newDecorator

Demonstration for @decorator:

deco = makeRegisteringDecorator(deco)

class Test2(object):
    @deco
    def method(self):
        pass

    @deco2()
    def method2(self):
        pass

def methodsWithDecorator(cls, decorator):
    """ 
        Returns all methods in CLS with DECORATOR as the
        outermost decorator.

        DECORATOR must be a "registering decorator"; one
        can make any decorator "registering" via the
        makeRegisteringDecorator function.
    """
    for maybeDecorated in cls.__dict__.values():
        if hasattr(maybeDecorated, 'decorator'):
            if maybeDecorated.decorator == decorator:
                print(maybeDecorated)
                yield maybeDecorated

It works!:

>>> print(list(   methodsWithDecorator(Test2, deco)   ))
[<function method at 0x7d62f8>]

However, a "registered decorator" must be the outermost decorator, otherwise the .decorator attribute annotation will be lost. For example in a train of

@decoOutermost
@deco
@decoInnermost
def func(): ...

you can only see metadata that decoOutermost exposes, unless we keep references to "more-inner" wrappers.

sidenote: the above method can also build up a .decorator that keeps track of the entire stack of applied decorators and input functions and decorator-factory arguments. =) For example if you consider the commented-out line R.original = func, it is feasible to use a method like this to keep track of all wrapper layers. This is personally what I'd do if I wrote a decorator library, because it allows for deep introspection.

There is also a difference between @foo and @bar(...). While they are both "decorator expressons" as defined in the spec, note that foo is a decorator, while bar(...) returns a dynamically-created decorator, which is then applied. Thus you'd need a separate function makeRegisteringDecoratorFactory, that is somewhat like makeRegisteringDecorator but even MORE META:

def makeRegisteringDecoratorFactory(foreignDecoratorFactory):
    def newDecoratorFactory(*args, **kw):
        oldGeneratedDecorator = foreignDecoratorFactory(*args, **kw)
        def newGeneratedDecorator(func):
            modifiedFunc = oldGeneratedDecorator(func)
            modifiedFunc.decorator = newDecoratorFactory # keep track of decorator
            return modifiedFunc
        return newGeneratedDecorator
    newDecoratorFactory.__name__ = foreignDecoratorFactory.__name__
    newDecoratorFactory.__doc__ = foreignDecoratorFactory.__doc__
    return newDecoratorFactory

Demonstration for @decorator(...):

def deco2():
    def simpleDeco(func):
        return func
    return simpleDeco

deco2 = makeRegisteringDecoratorFactory(deco2)

print(deco2.__name__)
# RESULT: 'deco2'

@deco2()
def f():
    pass

This generator-factory wrapper also works:

>>> print(f.decorator)
<function deco2 at 0x6a6408>

bonus Let's even try the following with Method #3:

def getDecorator(): # let's do some dispatching!
    return deco

class Test3(object):
    @getDecorator()
    def method(self):
        pass

    @deco2()
    def method2(self):
        pass

Result:

>>> print(list(   methodsWithDecorator(Test3, deco)   ))
[<function method at 0x7d62f8>]

As you can see, unlike method2, @deco is correctly recognized even though it was never explicitly written in the class. Unlike method2, this will also work if the method is added at runtime (manually, via a metaclass, etc.) or inherited.

Be aware that you can also decorate a class, so if you "enlighten" a decorator that is used to both decorate methods and classes, and then write a class within the body of the class you want to analyze, then methodsWithDecorator will return decorated classes as well as decorated methods. One could consider this a feature, but you can easily write logic to ignore those by examining the argument to the decorator, i.e. .original, to achieve the desired semantics.

Answer 2

To expand upon @ninjagecko's excellent answer in Method 2: Source code parsing, you can use the ast module introduced in Python 2.6 to perform self-inspection as long as the inspect module has access to the source code.

def findDecorators(target):
    import ast, inspect
    res = {}
    def visit_FunctionDef(node):
        res[node.name] = [ast.dump(e) for e in node.decorator_list]

    V = ast.NodeVisitor()
    V.visit_FunctionDef = visit_FunctionDef
    V.visit(compile(inspect.getsource(target), '?', 'exec', ast.PyCF_ONLY_AST))
    return res

I added a slightly more complicated decorated method:

@x.y.decorator2
def method_d(self, t=5): pass

Results:

> findDecorators(A)
{'method_a': [],
 'method_b': ["Name(id='decorator1', ctx=Load())"],
 'method_c': ["Name(id='decorator2', ctx=Load())"],
 'method_d': ["Attribute(value=Attribute(value=Name(id='x', ctx=Load()), attr='y', ctx=Load()), attr='decorator2', ctx=Load())"]}

Answer 3

Maybe, if the decorators are not too complex (but I don't know if there is a less hacky way).

def decorator1(f):
    def new_f():
        print "Entering decorator1", f.__name__
        f()
    new_f.__name__ = f.__name__
    return new_f

def decorator2(f):
    def new_f():
        print "Entering decorator2", f.__name__
        f()
    new_f.__name__ = f.__name__
    return new_f


class A():
    def method_a(self):
      pass

    @decorator1
    def method_b(self, b):
      pass

    @decorator2
    def method_c(self, t=5):
      pass

print A.method_a.im_func.func_code.co_firstlineno
print A.method_b.im_func.func_code.co_firstlineno
print A.method_c.im_func.func_code.co_firstlineno

Answer 4

A simple way to solve this problem is to put code in the decorator that adds each function/method, that is passed in, to a data set (for example a list).

e.g.

def deco(foo):
    functions.append(foo)
    return foo

now every function with the deco decorator will be added to functions.

Answer 5

I don't want to add much, just a simple variation of ninjagecko's Method 2. It works wonders.

Same code, but using list comprehension instead of a generator, which is what I needed.

def methodsWithDecorator(cls, decoratorName):

    sourcelines = inspect.getsourcelines(cls)[0]
    return [ sourcelines[i+1].split('def')[1].split('(')[0].strip()
                    for i, line in enumerate(sourcelines)
                    if line.split('(')[0].strip() == '@'+decoratorName]