Using a command-line option in a pytest skip-if condition

Question

Long story short, I want to be able to skip some tests if the session is being run against our production API. The environment that the tests are run against is set with a c

Answer 1

Looks like true way to Control skipping of tests according to command line option is mark tests as skip dynamically:

1. add option using pytest_addoption hook like this:

def pytest_addoption(parser):
    parser.addoption(
        "--runslow", action="store_true", default=False, help="run slow tests"
    )

2. Use pytest_collection_modifyitems hook to add marker like this:

def pytest_collection_modifyitems(config, items):
    if config.getoption("--runslow"):
        # --runslow given in cli: do not skip slow tests
        return
    skip_slow = pytest.mark.skip(reason="need --runslow option to run")
    for item in items:
        if "slow" in item.keywords:
            item.add_marker(skip_slow)

3. Add mark to you test:

@pytest.mark.slow
def test_func_slow():
    pass

If you want to use the data from the CLI in a test, for example, it`s credentials, enough to specify a skip option when retrieving them from the pytestconfig:

1. add option using pytest_addoption hook like this:

def pytest_addoption(parser):
    parser.addoption(
        "--credentials",
        action="store",
        default=None,
        help="credentials to ..."
    )

2. use skip option when get it from pytestconfig

@pytest.fixture(scope="session")
def super_secret_fixture(pytestconfig):
    credentials = pytestconfig.getoption('--credentials', skip=True)
    ...

3. use fixture as usual in you test:

def test_with_fixture(super_secret_fixture):
    ...

In this case you will got something like this it you not send --credentials option to CLI:

Skipped: no 'credentials' option found

It is better to use _pytest.config.get_config instead of deprecated pytest.config If you still wont to use pytest.mark.skipif like this:

@pytest.mark.skipif(not _pytest.config.get_config().getoption('--credentials'), reason="--credentials was not specified")

Answer 2

The problem with putting global code in fixtures is that markers are evaluated before fixtures, so when skipif is evaluated, configInfo didn't run yet and pytest.global_env will be empty. I'd suggest to move the configuration code from the fixture to pytest_configure hook:

# conftest.py
import configparser
import pytest


def pytest_addoption(parser):
    parser.addoption('--ENV')


def pytest_configure(config):
    environment = config.getoption('--ENV')
    pytest.global_env = environment
    ...

The configuration hook is guaranteed to execute before the tests are collected and the markers are evaluated.

Is there a better way to be trying this than the pytest_namespace?

Some ways I know of:

1. Simply assign a module variable in pytest_configure (pytest.foo = 'bar', like I did in the example above).

2. Use the config object as it is shared throughout the test session:

   def pytest_configure(config):
       config.foo = 'bar'
   
   @pytest.fixture
   def somefixture(pytestconfig):
       assert pytestconfig.foo == 'bar'
   
   def test_foo(pytestconfig):
       assert pytestconfig.foo == 'bar'
   

   Outside of the fixtures/tests, you can access the config via pytest.config, for example:

   @pytest.mark.skipif(pytest.config.foo == 'bar', reason='foo is bar')
   def test_baz():
       ...
   

3. Use caching; this has an additional feature of persisting data between the test runs:

   def pytest_configure(config):
       config.cache.set('foo', 'bar')
   
   @pytest.fixture
   def somefixture(pytestconfig):
       assert pytestconfig.cache.get('foo', None)
   
   def test_foo(pytestconfig):
       assert pytestconfig.cache.get('foo', None)
   
   @pytest.mark.skipif(pytest.config.cache.get('foo', None) == 'bar', reason='foo is bar')
   def test_baz():
       assert True
   

When using 1. or 2., make sure you don't unintentionally overwrite pytest stuff with your own data; prefixing your own variables with a unique name is a good idea. When using caching, you don't have this problem.