Dynamic importing of modules followed by instantiation of objects with a certain baseclass from said modules

Question

I\'m writing an application. No fancy GUI:s or anything, just a plain old console application. This application, lets call it App, needs to be able to load plugins on startu

Answer 1

Could you use execfile() instead of import with a specified namespace dict, then iterate over that namespace with issubclass, etc?

Answer 2

You would make this a lot easier if you forced some constraints on the plugin writer, for example that all plugins must be packages that contain a load_plugin( app, config) function that returns a Plugin instance. Then all you have to do is try to import these packages and run the function.

Answer 3

Here is a meta-classier way to register the plugins:

Define PluginBase to be of type PluginType. PluginType automatically registers any instance (class) in the plugins set.

plugin.py:

plugins=set()
class PluginType(type):
    def __init__(cls, name, bases, attrs):
        super(PluginType, cls).__init__(name, bases, attrs)
        # print(cls, name,cls.__module__)
        plugins.add(cls)

class PluginBase(object):
    __metaclass__=PluginType
    pass

This is the part that the user writes. Notice that there is nothing special here.

pluginDir/myplugin.py:

import plugin
class Foo(plugin.PluginBase):
    pass

Here is what the search function might look like:

test.py:

import plugin
import os
import imp

def search(plugindir):
    for root, dirs, files in os.walk(plugindir):
        for fname in files:
            modname = os.path.splitext(fname)[0]
            try:
                module=imp.load_source(modname,os.path.join(root,fname))
            except Exception: continue

search('pluginDir')
print(plugin.plugins)

Running test.py yields

set([<class 'myplugin.Foo'>])

Answer 4

You might do something like this:

for c in candidates:
    modname = os.path.splitext(c)[0]
    try:
        module=__import__(modname)   #<-- You can get the module this way
    except (ImportError,NotImplementedError):
        continue
    for cls in dir(module):          #<-- Loop over all objects in the module's namespace
        cls=getattr(module,cls)
        if (inspect.isclass(cls)                # Make sure it is a class 
            and inspect.getmodule(cls)==module  # Make sure it was defined in module, not just imported
            and issubclass(cls,base)):          # Make sure it is a subclass of base
            # print('found in {f}: {c}'.format(f=module.__name__,c=cls))
            classList.append(cls)

To test the above, I had to modify your code a bit; below is the full script.

import sys
import inspect
import os

class PluginBase(object): pass

def search(base):
    for root, dirs, files in os.walk('.'):
        candidates = [fname for fname in files if fname.endswith('.py') 
                      and not fname.startswith('__')]
        classList=[]
        if candidates:
            for c in candidates:
                modname = os.path.splitext(c)[0]
                try:
                    module=__import__(modname)
                except (ImportError,NotImplementedError):
                    continue
                for cls in dir(module):
                    cls=getattr(module,cls)
                    if (inspect.isclass(cls)
                        and inspect.getmodule(cls)==module
                        and issubclass(cls,base)):
                        # print('found in {f}: {c}'.format(f=module.__name__,c=cls))
                        classList.append(cls)
        print(classList)

search(PluginBase)