Add information to every log message in Python logging

Question

I am using Python with logging module and would like to add the socket.hostname() to every log message, I have to run this query every message and can not use

Answer 1

This builds upon the answer by Philippe while using dictConfig. The contextual filter demonstrated in this answer uses psutil to log the current CPU and memory usage percentage in each log message.

Save this file in say mypackage/util/logging.py:

"""logging utiliies."""
import logging

from psutil import cpu_percent, virtual_memory


class PsutilFilter(logging.Filter):
    """psutil logging filter."""

    def filter(self, record: logging.LogRecord) -> bool:
        """Add contextual information about the currently used CPU and virtual memory percentages into the given log record."""
        record.psutil = f"c{cpu_percent():02.0f}m{virtual_memory().percent:02.0f}"  # type: ignore
        return True

Note that a filter function didn't work for me; only a filter class worked.

Next, update your logging config dict based on this answer as below:

LOGGING_CONFIG = {
    ...,
    "filters": {"psutil": {"()": "mypackage.util.logging.PsutilFilter"}},
    "handlers": {"console": {..., "filters": ["psutil"]}},
    "formatters": {
        "detailed": {
            "format": "%(asctime)s %(levelname)s %(psutil)s %(process)x:%(threadName)s:%(name)s:%(lineno)d:%(funcName)s: %(message)s"
        }
    },
}

Try logging something, and see sample output such as:

2020-05-16 01:06:08,973 INFO c68m51 3c:MainThread:mypackage.mymodule:27:myfunction: This is my log message.

In the above message, c68m51 means 68% CPU usage and 51% virtual memory usage.