AWS Elastic Beanstalk logging with python (django)

Question

How do you manage your application logs in AWS elastic beanstalk? Which file you write you application logs to?

I\'m using the following Logging configuration in my d

Answer 1

There's a simple way that doesn't require any beanstalk configuration.

In your django settings under LOGGING set up a handler directed to the file '/opt/python/log/{log_file_name}'. The logs can then be accessed via the beanstalk environment menu under "Logs".

LOGGING = {
    ...,
    'handlers': {
        'logfile': {
            'level': 'DEBUG',
            'class': 'logging.handlers.RotatingFileHandler',
            'filename': '/opt/python/log/{log_file_name}',
        },
    },
    'loggers': {
        'debugger': {
            'level': 'DEBUG',
            'handlers': ['logfile'],
        'propagate': False,
    },
}

This location is stated in the documentation here:

https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.logging.html#health-logs-instancelocation

Answer 2

--edit--

This answer was originally written for Amazon Linux AMI. On Amazon Linux AMI, it is convenient to use /opt/python/log, because all log files in the /opt/python/log folder are automatically included when you request logs from Elastic Beanstalk. This is because the default configuration contains a task, located in /opt/elasticbeanstalk/tasks, that includes all files in /opt/python/log. The official AWS Python sample-application for Amazon Linux AMI also uses /opt/python/log for logging.

However, on Amazon Linux 2, there is no /opt/python/log folder (as pointed out by @Oded in the comments). So, now what do we do?

The official AWS Python sample-application for Amazon linux 2 uses the /tmp folder for logging, but custom log files added to /tmp are not included automatically when requesting logs from Elastic Beanstalk. The sample application now uses custom logging tasks, located in /opt/elasticbeanstalk/tasks subfolders. See instructions in the documentation. The sample app source shows how this is accomplished using .ebextensions:

files:
  "/opt/elasticbeanstalk/tasks/bundlelogs.d/01-sample-app.conf":
    content: |
      /tmp/sample-app*

  "/opt/elasticbeanstalk/tasks/taillogs.d/01-sample-app.conf":
    content: |
      /tmp/sample-app.log

This implies that the convenience of using /opt/python/log, without the need for any custom configuration, is lost. Nevertheless, some of the permission issues, described in the original answer below, might still arise.

The default file permissions for /tmp/sample-app.log, on a test-instance running the default Python sample application on Amazon Linux 2, with all default settings, are as follows:

-rw-r--r-- 1 webapp webapp 0 Aug 7 14:24 sample-app.log

Note that owner and group are now called webapp.

Note that the application itself can now be found in /var/app/current.

See AWS Linux 2 migration docs and extending EB Linux platforms for more information.

--original--

Summary

The simplest solution, in my opinion, is to log to the /opt/python/log folder, as suggested by bewestphal and by @thierry-j (under steve-dunlop's answer).

That is also what the official AWS EB Python Sample Application does: see python-v1.zip source

The log file will then be included automatically when you request logs from EB.

This solution works out-of-the-box, without any modification of .ebextensions, as long as you do not call django-admin.py (or other django code) in your .ebextensions.

However, most apps do need to call django-admin.py in .ebextensions, e.g. in order to migrate. That will cause the log file to be created prematurely, with root owner and root group. This leads to permission errors, because the app runs as wsgi:wsgi.

This can be fixed by adding a new command, at the end of your container_commands, to remove the "premature" log file, e.g.:

container_commands:
  ...
  9999_remove_root_log_file:
    command: rm /opt/python/log/django.log
    ignoreErrors: true

Details below.

Background

On a standard pre-configured Amazon Linux/Python platform, which uses Apache with mod_wsgi (see AWS platform docs), the WSGIDaemonProcess for the Django app runs as user wsgi and group wsgi (see /etc/httpd/conf.d/wsgi.conf on your EC2 instance).

In addition, the default folder permissions for the /opt/python/log folder (on my standard EC2 instance) are: drwxrwxr-x 3 root wsgi 4096 Mar 5 14:08 .

That is, the wsgi group has all permissions (rwx), so the Django app (group wsgi) can create log files there.

This works, out-of-the-box, as demonstrated by the official AWS EB Python Sample Application (python-v1.zip).

However, if you do anything in your .ebextensions that causes the logging file-handler to be initialized (like calling django-admin.py), it will break.

Permission issues

Here's how using django-admin.py in .ebextensions breaks your log file permissions:

Elastic Beanstalk container_commands, in .ebextensions, are executed as the root user (see aws docs).

If you call django-admin.py in any of the container_commands, e.g. with collectstatic or migrate, that will cause your logging file handler(s) to be initialized. If the specified log file does not exist yet, at that time, it will be created, with root owner and root group.

That means the Django app, running as part of the wsgi group, will not have permission to write to the log file (which belongs to the root group).

This leads to permission errors, e.g.: PermissionError: [Errno 13] Permission denied: '/opt/python/log/django.log'

How to reproduce

The following snippet illustrates the permissions issue and shows how to fix it.

To reproduce the issue, add these container_commands to a clean project (e.g. following the AWS EB Django tutorial), configure Django settings.py to log to /opt/python/log/django.log, deploy to AWS EB, then check the eb-activity.log to see the output of the container commands.

...

container_commands:
  0100_show_current_user:
    # show that we are running as root user
    command: whoami
  0200_try_to_remove_log_file:
    # we need a clean slate for this example (make sure no log file owned by wsgi is present)
    command: rm /opt/python/log/django.log
    ignoreErrors: true
  0300_break_log_file_permissions:
    # this causes a new log file to be created, owned by root:root (instead of wsgi:wsgi)
    command: django-admin.py
  0400_show_log_file_permissions:
    # prove that a log file was created by root, and show folder permissions
    command: ls -la /opt/python/log
  0500_fix_by_removing_log_file_after_all_django_admin_calls:
    # remove the log file created by django-admin.py, to ensure that a new log file will  
    # be created when the server starts, owned by wsgi:wsgi
    command: rm /opt/python/log/django.log
    ignoreErrors: true

DRY solution

So, there is no need to mess with file/folder permissions explicitly.

If you don't call django code in .ebextensions, logging to /opt/python/log works, out-of-the-box.

If you do call django code in .ebextensions, e.g. django-admin.py collectstatic, simply remove the log file at the end of your container_commands section.

Here's a DRY example:

In .ebextensions config:

option_settings:
  # create EB environment property for the log file path
  aws:elasticbeanstalk:application:environment:
    LOG_FILE_PATH: /opt/python/log/django.log
...

container_commands:
  ...
  # django code called here, e.g. "django-admin.py collectstatic"
  ...
  9999_remove_any_existing_django_log_files:
    command: rm $LOG_FILE_PATH      
    ignoreErrors: true

and in settings.py:

...
# get log path from environment variable, with fallback for local development
log_file_path = os.getenv('LOG_FILE_PATH', 'local.log')
# use this as 'filename' for the file handler, as described in the other answers
...

Answer 3

By default in elasticbeanstalk, you can see the django error logs here.

/var/log/httpd/error_log

Answer 4

None of the mentioned solutions worked for me as I am running my server on Amazon Linux 2.

The problem wasn't in the logging configuration itself but in the .log file permissions.

This is my related logger.config file in .ebextensions:

commands:
  01_create_log_file:
    command: 'touch /var/log/django.log'
  02_change_log_file_permissions:
    command: 'sudo chmod ugo+rwx /var/log/django.log'

As you can see I had to handle the permission problem by simply giving all permissions to all users.

This is my Django logger configuration:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'file': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename': '/var/log/django.log',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['file'],
            'level': 'DEBUG',
            'propagate': True,
        },
    },
}

Answer 5

As a beginner in terms of linux permissions, it took me some time to get it to work. Summarising the above given answers the following finally worked for me:

logging.config

commands:
  00_create_dir:
    command: mkdir -p /var/log/app-logs
  01_change_permissions:
    command: chmod g+s /var/log/app-logs
  02_change_default_owner:
    command: setfacl -d -m g::rw /var/log/app-logs
  03_change_owner:
    command: chown wsgi:wsgi /var/log/app-logs

settings.py

LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
    'file': {
        'level': 'DEBUG',
        'class': 'logging.FileHandler',
        'filename': '/var/log/app-logs/django.log',
    },
},
'loggers': {
    'django': {
        'handlers': ['file'],
        'level': 'DEBUG',
        'propagate': True,
    },
},
}

With this I can see the logs as a separate section using 'eb logs' or within the Beanstalk environment, section "logs".

Answer 6

Ok, I figured out a way to do it.

First I connected via ssh to ec2 machine, then I create a folder in /var/log called app_logs with root user:

mkdir /var/log/app_logs

After that I did the follow:

cd /var/log/
chmod g+s app_logs/
setfacl -d -m g::rw app_logs/
chown wsgi:wsgi app_logs/

That ensures that all the files created in this folder will have wsgi as owner and will be writable for the group that the file belongs. I had to do that because I noticed that the log file created by django app had root as owner and owner group but the application runs through wsgi user.

Finally I changed DEBUG_LOG_DIR to /var/log/app_logs/django_debug.log