Setting options from environment variables when using argparse

Question

I have a script which has certain options that can either be passed on the command line, or from environment variables. The CLI should take precedence if both are present, a

Answer 1

There is an example use-case for ChainMap where you merge together defaults, environment variables and command line arguments.

import os, argparse

defaults = {'color': 'red', 'user': 'guest'}

parser = argparse.ArgumentParser()
parser.add_argument('-u', '--user')
parser.add_argument('-c', '--color')
namespace = parser.parse_args()
command_line_args = {k:v for k, v in vars(namespace).items() if v}

combined = ChainMap(command_line_args, os.environ, defaults)

Came to me from a great talk about beautiful and idiomatic python.

However, I'm not sure how to go about the difference of lower- and uppercase dictionary keys. In the case where both -u foobar is passed as an argument and environment is set to USER=bazbaz, the combined dictionary will look like {'user': 'foobar', 'USER': 'bazbaz'}.

Answer 2

The topic is quite old, but I had similar problem and I thought I would share my solution with you. Unfortunately custom action solution suggested by @Russell Heilling doesn't work for me for couple of reasons:

• It prevents me from using predefined actions (like store_true)
• I would rather like it to fallback to default when envvar is not in os.environ (that could be easily fixed)
• I would like to have this behaviour for all of my arguments without specifying action or envvar (which should always be action.dest.upper())

Here's my solution (in Python 3):

class CustomArgumentParser(argparse.ArgumentParser):
    class _CustomHelpFormatter(argparse.ArgumentDefaultsHelpFormatter):
        def _get_help_string(self, action):
            help = super()._get_help_string(action)
            if action.dest != 'help':
                help += ' [env: {}]'.format(action.dest.upper())
            return help

    def __init__(self, *, formatter_class=_CustomHelpFormatter, **kwargs):
        super().__init__(formatter_class=formatter_class, **kwargs)

    def _add_action(self, action):
        action.default = os.environ.get(action.dest.upper(), action.default)
        return super()._add_action(action)

Answer 3

The Click library handles this explicitly:

import click

@click.command()
@click.argument('src', envvar='SRC', type=click.File('r'))
def echo(src):
    """Print value of SRC environment variable."""
    click.echo(src.read())

And from the command line:

$ export SRC=hello.txt
$ echo
Hello World!

https://click.palletsprojects.com/en/master/arguments/#environment-variables

You can install it with

pip install click

Answer 4

Another option:

    parser = argparse.ArgumentParser()
    env = os.environ
    def add_argument(key, *args, **kwargs):
        if key in env:
            kwargs['default'] = env[key]
        parser.add_argument(*args, **kwargs)

    add_argument('--type', type=str)

Or this one, using os.getenv for setting default value:

parser = argparse.ArgumentParser()
parser.add_argument('--type', type=int, default=os.getenv('type',100))

Answer 5

I usually have to do this for multiple arguments (authentication and API keys).. this is simple and straight forward. Uses **kwargs.

def environ_or_required(key):
    return (
        {'default': os.environ.get(key)} if os.environ.get(key)
        else {'required': True}
    )

parser.add_argument('--thing', **environ_or_required('THING'))

Answer 6

Thought I'd post my solution as the original question/answer gave me a lot of help.

My problem is a little different to Russell's. I'm using OptionParser and instead of an environmental variable for each argument I have just one which simulates the command line.

i.e.

MY_ENVIRONMENT_ARGS = --arg1 "Maltese" --arg2 "Falcon" -r "1930" -h

Solution:

def set_defaults_from_environment(oparser):

    if 'MY_ENVIRONMENT_ARGS' in os.environ:

        environmental_args = os.environ[ 'MY_ENVIRONMENT_ARGS' ].split()

        opts, _ = oparser.parse_args( environmental_args )

        oparser.defaults = opts.__dict__

oparser = optparse.OptionParser()
oparser.add_option('-a', '--arg1', action='store', default="Consider")
oparser.add_option('-b', '--arg2', action='store', default="Phlebas")
oparser.add_option('-r', '--release', action='store', default='1987')
oparser.add_option('-h', '--hardback', action='store_true', default=False)

set_defaults_from_environment(oparser)

options, _ = oparser.parse_args(sys.argv[1:])

Here I don't throw an error if an argument is not found. But if I wish to I could just do something like

for key in options.__dict__:
    if options.__dict__[key] is None:
        # raise error/log problem/print to console/etc