pip requirement.txt conditionals or environment markers based on env variables

问题

Is there a way to specify conditional installs in a pip requirements.txt file that are based on the value of an environment variable?

I've been able to control most of what I need with environment markers, but all the markers I know of work only with specific values that are essentially pre-defined by python.

For example, I want to be able to control package installation for RHEL 5.3 vs. RHEL 6.3 vs. RHEL 6.6, etc. Also based on other criteria.

It would be perfect if I could specify in the results.txt file an environment variable that would be checked against a value I set before running pip. This seems like it would be desirable and straight forward functionality. I haven't had much luck so far finding comprehensive discussions of environment markers, so I'm hoping I've just missed some key bit of information :-)

Thanks much.

回答1:

There's not really a way to do it with environment variables. Pip requirements files are basically just a list of pip install arguments listed in a file. So if your requirements file looks like this:

Foo==1.1.0
Bar==0.1.0
Baz==2.0.1

Logically, pip is doing this:

pip install Foo==1.1.0
pip install Bar==0.1.0
pip install Baz==2.0.1

Unfortunately, in that context there's no way to apply environment variables.

There are a couple solutions to this problem.

One, you could have a base requirements file, say requirements.txt, that lists common dependencies for all platforms. Then, you could have individual requirements files that are named, say, requirements.rhel53.txt, requirements.rhel63.txt, etc. The top of each of these files could have this as the first line:

-r requirements.txt

And then additional special dependencies listing. Then, in each environment, you could set an env var, let's call it $PLATFORM, and run a command like this to install dependencies:

$ pip install -r requirements.$PLATFORM.txt

Or, you could use constraints files. Your requirements.txt would just list dependencies without versions:

Foo
Bar
Baz

And then you could have a constraints file, again for each platform, that would list specific version requirements. For example, you could have constraints.rhel53.txt that had this:

Foo==1.1.0
Bar==0.1.0
Baz==2.0.1

And again, you set an env var and then run a command like this:

$ pip install -r requirements.txt -c constraints.$PLATFORM.txt

It's a cumbersome solution, but that would be one way of dealing with it. Unfortunately, pip does not have a native solution.

回答2:

I ended up with a scripted solution similar to one I found in: Is there a way to have a conditional requirements.txt file for my Python application based on platform?

Basically, I did something like the following (partial) example, which allows me to define packages as needed in the script, and still pass in requirements.txt and constraints.txt files. I'm new to Python, so please forgive (or comment on) any less-than-optimal coding practices:

import pip, sys, getopt

def install(options, packages):
    pipargs = options + packages
    pip.main(pipargs)

constraints_file = ''
requirements_file = ''

try:
    opts, args = getopt.getopt(sys.argv[1:],"hr:c:",["help","requirements-file=","constraints-file="])
except getopt.GetoptError:
    print_usage()
    sys.exit(2)
for opt, arg in opts:
    if opt in ("-h", "--help"):
       print_usage()
       sys.exit()
    elif opt in ("-r", "--requirements-file"):
        requirements_file = arg
    elif opt in ("-c", "--constraints-file"):
        constraints_file = arg

base_pkgs = [
    "nose>=1.3.4",
    "wheel",
    "pytz"
]

foo_pkgs = [
    "Pygments; python_version == '2.7'",
    "rednose; python_version == '3.5'",
]

bar_pkgs = [
    "six",
]

if __name__ == '__main__':

    from os import environ as env

    myvar = env.get('MYVAR')
    if(myvar == 'foo'):
        mypkgs = foo_pkgs
    elif(myvar == 'bar'):
        mypkgs = bar_pkgs
    else:
        mypkgs = ''

    pkglist = base_pkgs + mypkgs

    pipoptions = [  'install',
                    '--upgrade',
                    '--force-reinstall'
                 ]

    if(constraints_file):
        pipoptions += ['-c', constraints_file]
    if(requirements_file):
        pipoptions += ['-r', requirements_file]

    install(pipoptions, pkglist)

来源：https://stackoverflow.com/questions/35353489/pip-requirement-txt-conditionals-or-environment-markers-based-on-env-variables

标签

python

pip

requirements.txt