Use of external modules in a logging configuration file

Question

I have setup the following configuration file for the logging module in python2.7 with logstash

[loggers]
keys=root,test

[handlers]
keys=remote

[formatters]
keys=standard

[logger_root]
level=NOTSET
handlers=remote

[logger_test]
level=DEBUG
handlers=remote
propagate=0
qualname=test

[handler_remote]
class=logstash.LogstashHandler
level=NOTSET
formatter=standard
args=(os.environ['hostname'], int(os.environ['port']))

[formatter_standard] 
format=%(levelname)s - %(message)s
datefmt=
class=logging.Formatter

Unfortunately, this is about as short as I can make the file for this example. I use a module called logstash which sends my logging messages out to be viewed remotely. If I try and load this configuration with logging.config.fileConfig I get an error where it cannot find logstash. All I have to do is import logstash before loading and the problem goes away.

It would be nice if I could fit that import statement into the configuration file itself though, so that the loader of the file would not need to know or care. It would also be nice for things like urlparse so I wouldn't need as many environment variables.

Is there a way to do this?

Answer 1

Well this is quite an old question, but if anyone is still looking for an answer, I post this (available for both python2 and python3).

Short answer

According to the documentation, you can write something like:

class=ext://logstash.LogstashHandler

I'm not sure whether it's necessary or not to import logstash in the script parsing the logging configuration file (as logstash does not belong to the standard library).

Alternative answer

But I must add that this mechanism hasn't always worked very well for me in old-style logging configuration files... so if this doesn't work, I would recommend to turn your configuration file into a yaml one and use this very same feature in it:

handlers:
    remote:
        class: ext://logstash.LogstashHandler
        level: NOTSET
        formatter: standard

For the other args, I don't know how to write them since I can't find a documentation about logstash. If they are positional arguments, this should be OK (though I'm not sure about the use of int in a logging configuration file):

        args: [os.environ['hostname'], int(os.environ['port'])]

But if they are keyword arguments, this should be written another way:

        kw1: os.environ['hostname']
        kw2: int(os.environ['port'])