storing logger messages in a string

Question

I wanted to store all the intermediate log messages (warn, info, error) to a string in python and finally at the end of program, display everything as a report to the consol

Answer 1

It can be as simple as logging to a StringIO object:

import logging
try:
    from cStringIO import StringIO      # Python 2
except ImportError:
    from io import StringIO

log_stream = StringIO()    
logging.basicConfig(stream=log_stream, level=logging.INFO)

logging.info('hello world')
logging.warning('be careful!')
logging.debug("you won't see this")
logging.error('you will see this')
logging.critical('critical is logged too!')

print(log_stream.getvalue())

Output

INFO:root:hello world
WARNING:root:be careful!
ERROR:root:you will see this
CRITICAL:root:critical is logged too!

---

If you want to log only those messages at levels WARN, INFO and ERROR you can do it with a filter. LevelFilter below checks each log record's level no, allowing only those records of the desired level(s):

import logging
try:
    from cStringIO import StringIO      # Python 2
except ImportError:
    from io import StringIO

class LevelFilter(logging.Filter):
    def __init__(self, levels):
        self.levels = levels

    def filter(self, record):
        return record.levelno in self.levels

log_stream = StringIO()    
logging.basicConfig(stream=log_stream, level=logging.NOTSET)
logging.getLogger().addFilter(LevelFilter((logging.INFO, logging.WARNING, logging.ERROR)))

logging.info('hello world')
logging.warning('be careful!')
logging.debug("you won't see this")
logging.error('you will see this')
logging.critical('critical is no longer logged!')

print(log_stream.getvalue())

Output

INFO:root:hello world
WARNING:root:be careful!
ERROR:root:you will see this

Answer 2

Note that solutions involving basicConfig set attributes of the root logger which all other loggers inherit from, this can be unwanted because libraries will also log to it. My use case is a website that calls a data processing module, and I only want to capture that module's logs specifically. This also has the advantage of allowing existing handlers that log to file and the terminal to persist:

import io, logging
from django.http import HttpResponse

log_stream = io.StringIO()
log_handler = logging.StreamHandler(log_stream)
logging.getLogger('algorithm.user_output').addHandler(log_handler)

algorithm()
return HttpResponse(f'<pre>{log_stream.getvalue()}</pre>')

In algorithm.py:

logger = logging.getLogger(__name__ + '.user_output')  # 'algorithm.user_output'

Answer 3

Maybe this example code is enough.
In general, you should post your code so we can see what is going on.
You should also be looking at the actual Python documentation for the logging module while you are following any given tutorial.
https://docs.python.org/2/library/logging.html

The standard Python logging module can log to a file. When you are done logging, you can print the contents of that file to your shell output.

# Do some logging to a file
fname = 'mylog.log'
logging.basicConfig(filename=fname, level=logging.INFO)
logging.info('Started')
logging.info('Finished')

# Print the output
with open(fname, 'r') as f:
    print f.read() # You could also store f.read() to a string

Answer 4

You can also write your own stream class. As https://docs.python.org/2/library/logging.handlers.html says, only writeand flushare used for the streaming.

Example:

import logging

class LogStream(object):
    def __init__(self):
        self.logs = ''

    def write(self, str):
        self.logs += str

    def flush(self):
        pass

    def __str__(self):
        return self.logs

log_stream = LogStream()
logging.basicConfig(stream=log_stream, level=logging.DEBUG)

log = logging.getLogger('test')
log.debug('debugging something')
log.info('informing user')

print(log_stream)

Outputs:

DEBUG:test:debugging something
INFO:test:informing user