Add information to every log message in Python logging

Question

I am using Python with logging module and would like to add the socket.hostname() to every log message, I have to run this query every message and can not use

Answer 1

You can use filter to add information to every message :

import logging
import socket

class ContextFilter(logging.Filter):
    def filter(self, record):
        record.hostname = socket.gethostname() 
        return True

if __name__ == '__main__':
    levels = (logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR, logging.CRITICAL)
    logging.basicConfig(level=logging.DEBUG,
                        format='%(asctime)-15s hostname: %(hostname)-15s : %(message)s')
    a1 = logging.getLogger('a.b.c')
    f = ContextFilter()
    a1.addFilter(f)
    a1.debug('A debug message')

Answer 2

You can configure the logging module by adding a custom format option like so

import logging

name = socket.hostname() 
logMessageFormat = '{}: %(levelname)s:%(message)s'.format(name)
logging.basicConfig(format=logMessageFormat, level=logging.DEBUG)

# Test new configuration
logger = logging.getLogger()
logger.info('Hello world')

# should print to the console
# <socketHostName>: INFO:Hello world

You can read more about customizing the format of displayed messages here https://docs.python.org/3/howto/logging.html#changing-the-format-of-displayed-messages

Answer 3

This builds upon the answer by Philippe while using dictConfig. The contextual filter demonstrated in this answer uses psutil to log the current CPU and memory usage percentage in each log message.

Save this file in say mypackage/util/logging.py:

"""logging utiliies."""
import logging

from psutil import cpu_percent, virtual_memory


class PsutilFilter(logging.Filter):
    """psutil logging filter."""

    def filter(self, record: logging.LogRecord) -> bool:
        """Add contextual information about the currently used CPU and virtual memory percentages into the given log record."""
        record.psutil = f"c{cpu_percent():02.0f}m{virtual_memory().percent:02.0f}"  # type: ignore
        return True

Note that a filter function didn't work for me; only a filter class worked.

Next, update your logging config dict based on this answer as below:

LOGGING_CONFIG = {
    ...,
    "filters": {"psutil": {"()": "mypackage.util.logging.PsutilFilter"}},
    "handlers": {"console": {..., "filters": ["psutil"]}},
    "formatters": {
        "detailed": {
            "format": "%(asctime)s %(levelname)s %(psutil)s %(process)x:%(threadName)s:%(name)s:%(lineno)d:%(funcName)s: %(message)s"
        }
    },
}

Try logging something, and see sample output such as:

2020-05-16 01:06:08,973 INFO c68m51 3c:MainThread:mypackage.mymodule:27:myfunction: This is my log message.

In the above message, c68m51 means 68% CPU usage and 51% virtual memory usage.