Scrapy - logging to file and stdout simultaneously, with spider names

Question

I\'ve decided to use the Python logging module because the messages generated by Twisted on std error is too long, and I want to INFO level meaningful messages such

Answer 1

As of Scrapy 2.3, none of the answers mentioned above worked for me. In addition, the solution found in the documentation caused overwriting of the log file with every message, which is of course not what you want in a log. I couldn't find a built-in setting that changed the mode to "a" (append). I achieved logging to both file and stdout with the following configuration code:

configure_logging(settings={
    "LOG_STDOUT": True
})
file_handler = logging.FileHandler(filename, mode="a")
formatter = logging.Formatter(
    fmt="%(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s",
    datefmt="%H:%M:%S"
)
file_handler.setFormatter(formatter)
file_handler.setLevel("DEBUG")
logging.root.addHandler(file_handler)