Python logger: won't overwrite the original log?

Question

So, when I copy paste the following x times to the python prompt, it add the log x times to the end of the designated file.

How can I change the code so that each ti

Answer 1

The mode is specified as part of logging.basicConfig and is passed through using filemode.

logging.basicConfig(
    level = logging.DEBUG,
    format = '%(asctime)s %(levelname)s %(message)s',
    filename = 'oneDayFileLoader.log,
    filemode = 'w'
)

https://docs.python.org/3/library/logging.html#simple-examples

Answer 2

Your log messages are being duplicated because you call addHandler more than once. Each call to addHandler adds an additional log handler.

If you want to make sure the file is created from scratch, add an extra line of code to remove it:

os.remove(os.path.join(paths["work"], "oneDayFileLoader.log"))

Answer 3

If I understand correctly, you're running a certain Python process for days at a time, and want to rotate the log every day. I'd recommend you go a different route, using a handler that automatically rotates the log file, e.g. http://www.blog.pythonlibrary.org/2014/02/11/python-how-to-create-rotating-logs/

But, if you want to control the log using the process in the same method you're comfortable with (Python console, pasting in code.. extremely unpretty and error prone, but sometimes quick-n-dirty is sufficient for the task at hand), well...

Your issue is that you create a new FileHandler each time you paste in the code, and you add it to the Logger object. You end up with a logger that has X FileHandlers attached to it, all of them writing to the same file. Try this:

import logging
paths = {'work': ''}
logger = logging.getLogger('oneDayFileLoader')
if logger.handlers:
   logger.handlers[0].close()
   logger.handlers = []
logHandler = logging.FileHandler(os.path.join(paths["work"] , "oneDayFileLoader.log"), mode='w')
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
logHandler.setFormatter(formatter)
logger.addHandler(logHandler) 
logger.setLevel(logging.DEBUG)
logger.error('overwrite')

Based on your request, I've also added an example using TimedRotatingFileHandler. Note I haven't tested it locally, so if you have issues ping back.

import logging
from logging.handlers import TimedRotatingFileHandler

logPath = os.path.join('', "fileLoaderLog")
logger = logging.getLogger('oneDayFileLoader')
logHandler = TimedRotatingFileHandler(logPath,
                                   when="midnight",
                                   interval=1)
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
logHandler.setFormatter(formatter)
logger.addHandler(logHandler) 
logger.setLevel(logging.DEBUG)
logger.error('overwrite')