Python logging and rotating files

Question

I have a python program that is writing to a log file that is being rotated by Linux\'s logrotate command. When this happens I need to signal my program to stop writing to

Answer 1

Don't use logging.basicConfig, use WatchedFileHandler. Here's how to use it.

import time
import logging
import logging.handlers

def log_setup():
    log_handler = logging.handlers.WatchedFileHandler('my.log')
    formatter = logging.Formatter(
        '%(asctime)s program_name [%(process)d]: %(message)s',
        '%b %d %H:%M:%S')
    formatter.converter = time.gmtime  # if you want UTC time
    log_handler.setFormatter(formatter)
    logger = logging.getLogger()
    logger.addHandler(log_handler)
    logger.setLevel(logging.DEBUG)

log_setup()
logging.info('Hello, World!')
import os
os.rename('my.log', 'my.log-old')
logging.info('Hello, New World!')

Answer 2

Since rotation is already being done by logrotate, in your signal handler you should just call logging.basicConfig(...) again and that should reopen the log file.

Answer 3

You may want to look at WatchedFileHandler to implement this, or as an alternative, implement log rotation with RotatingFileHandler, both of which are in the logging.handlers module.

Answer 4

from logging import handlers

handler = handlers.TimedRotatingFileHandler(filename, when=LOG_ROTATE)

handler.setFormatter(logging.Formatter(log_format, datefmt="%d-%m-%Y %H:%M:%S"))

#LOG_ROTATE = midnight    
#set your log format

This should help you in handling rotating log