How do I write Flask's excellent debug log message to a file in production?

Question

I have a Flask application that works well and produces an occasional error, which is visible when it is running with debug=True:

if __name__ =         


        

Answer 1

I'm not a specialist on logging module, but regarding my experience on it + some years of on Python + Flask, you can have a good logging configuration, considering some observations:

• at the beginning of every function (route), create a timestamp object, in order to registry the exact time when the request was made, independently if it was successful or not

• use @app.after_request, for registering every successful request

• use @app.errorhandler, for registering general errors + Tracebacks

Here is an example that demonstrates this idea:

#/usr/bin/python3
""" Demonstration of logging feature for a Flask App. """

from logging.handlers import RotatingFileHandler
from flask import Flask, request, jsonify
from time import strftime

__author__ = "@ivanleoncz"

import logging
import traceback


app = Flask(__name__)

@app.route("/")
@app.route("/index")
def get_index():
    """ Function for / and /index routes. """
    return "Welcome to Flask! "


@app.route("/data")
def get_data():
    """ Function for /data route. """
    data = {
            "Name":"Ivan Leon",
            "Occupation":"Software Developer",
            "Technologies":"[Python, Flask, JavaScript, Java, SQL]"
    }
    return jsonify(data)


@app.route("/error")
def get_nothing():
    """ Route for intentional error. """
    return foobar # intentional non-existent variable


@app.after_request
def after_request(response):
    """ Logging after every request. """
    # This avoids the duplication of registry in the log,
    # since that 500 is already logged via @app.errorhandler.
    if response.status_code != 500:
        ts = strftime('[%Y-%b-%d %H:%M]')
        logger.error('%s %s %s %s %s %s',
                      ts,
                      request.remote_addr,
                      request.method,
                      request.scheme,
                      request.full_path,
                      response.status)
    return response


@app.errorhandler(Exception)
def exceptions(e):
    """ Logging after every Exception. """
    ts = strftime('[%Y-%b-%d %H:%M]')
    tb = traceback.format_exc()
    logger.error('%s %s %s %s %s 5xx INTERNAL SERVER ERROR\n%s',
                  ts,
                  request.remote_addr,
                  request.method,
                  request.scheme,
                  request.full_path,
                  tb)
    return "Internal Server Error", 500


if __name__ == '__main__':
    handler = RotatingFileHandler('app.log', maxBytes=10000, backupCount=3)        
    logger = logging.getLogger(__name__)
    logger.setLevel(logging.ERROR)
    logger.addHandler(handler)
    app.run(host="127.0.0.1",port=8000)

For more information regarding logrotate and logs on stdout and file at the same time: this Gist

Answer 2

If you are using gunicorn to run your Flask app, you can log all Flask exceptions to the gunicorn logs by adding the gunicorn error handlers to the Flask logger:

In module/__init__.py:

@app.before_first_request
def setup_logging():
    if not app.debug:
        import logging
        gunicorn_logger = logging.getLogger('gunicorn.error')
        for handler in gunicorn_logger.handlers:
            app.logger.addHandler(handler)

Answer 3

For those who read this later on.

I think it is better idea to push more useful info into error messages. URL, client IP, user-agent etc. Flask logs exceptions internally (in app.debug==False mode) with Flask.log_exception function. So, instead of logging things manually in @app.errorhandler I do something like this:

class MoarFlask(Flask):
    def log_exception(self, exc_info):
        """...description omitted..."""
        self.logger.error(
            """
Request:   {method} {path}
IP:        {ip}
User:      {user}
Agent:     {agent_platform} | {agent_browser} {agent_browser_version}
Raw Agent: {agent}
            """.format(
                method = request.method,
                path = request.path,
                ip = request.remote_addr,
                agent_platform = request.user_agent.platform,
                agent_browser = request.user_agent.browser,
                agent_browser_version = request.user_agent.version,
                agent = request.user_agent.string,
                user=user
            ), exc_info=exc_info
        )

Then, at configuration time, bind FileHandler to app.logger and go on. I don't use StreamHandler cause many servers (e.g. uWSGI) like to pollute it with their own proprietary-wordy-useless-not-turnable-off messages.

Don't be afraid of extending Flask. You'll be forced to do it sooner or later ;)

Answer 4

I don't know why it's not working but I can tell how am doing this.

First of all, you don't need to set the level of app.logger. So remove this line app.logger.setLevel().

You want to save exception and return error page for every view. It is a lot of work to write this code everywhere. Flask provides a method to do this. Define an errorhandler method like this.

    @app.errorhandler(500)
    def internal_error(exception):
        app.logger.error(exception)
        return render_template('500.html'), 500

Whenever a view raises an exception, this method will be called and passed the exception as argument. Python logging provides exception method that is used to save full traceback of the exception.

Since this handles all exception, you don't even need to put code in try/except block. Though, if you want to do something before calling the errorhandler(for e.g. rollback session or transaction) then do this:

    try:
        #code
    except:
        #code
        raise

If you would like the date and time added for each entry in your log file, the following code can be used (in place of the similar code featured in the question).

if app.debug is not True:   
    import logging
    from logging.handlers import RotatingFileHandler
    file_handler = RotatingFileHandler('python.log', maxBytes=1024 * 1024 * 100, backupCount=20)
    file_handler.setLevel(logging.ERROR)
    formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
    file_handler.setFormatter(formatter)
    app.logger.addHandler(file_handler)

Answer 5

In Development, make sure to set: app.config['PROPAGATE_EXCEPTIONS'] = False. Default is None: https://flask.palletsprojects.com/en/1.1.x/config/