Python CLI program unit testing

Question

I am working on a python Command-Line-Interface program, and I find it boring when doing testings, for example, here is the help information of the program:

Answer 1

Start from the user interface with functional tests and work down towards unit tests. It can feel difficult, especially when you use the argparse module or the click package, which take control of the application entry point.

The cli-test-helpers Python package has examples and helper functions (context managers) for a holistic approach on writing tests for your CLI. It's a simple idea, and one that works perfectly with TDD:

1. Start with functional tests (to ensure your user interface definition) and
2. Work towards unit tests (to ensure your implementation contracts)

Functional tests

NOTE: I assume you develop code that is deployed with a setup.py file or is run as a module (-m).

• Is the entrypoint script installed? (tests the configuration in your setup.py)
• Can this package be run as a Python module? (i.e. without having to be installed)
• Is command XYZ available? etc. Cover your entire CLI usage here!

Those tests are simplistic: They run the shell command you would enter in the terminal, e.g.

def test_entrypoint():
    exit_status = os.system('foobar --help')
    assert exit_status == 0

Note the trick to use a non-destructive operation (e.g. --help or --version) as we can't mock anything with this approach.

Towards unit tests

To test single aspects inside the application you will need to mimic things like command line arguments and maybe environment variables. You will also need to catch the exiting of your script to avoid the tests to fail for SystemExit exceptions.

Example with ArgvContext to mimic command line arguments:

@patch('foobar.command.baz')
def test_cli_command(mock_command):
    """Is the correct code called when invoked via the CLI?"""
    with ArgvContext('foobar', 'baz'), pytest.raises(SystemExit):
        foobar.cli.main()

    assert mock_command.called

Note that we mock the function that we want our CLI framework (click in this example) to call, and that we catch SystemExit that the framework naturally raises. The context managers are provided by cli-test-helpers and pytest.

Unit tests

The rest is business as usual. With the above two strategies we've overcome the control a CLI framework may have taken away from us. The rest is usual unit testing. TDD-style hopefully.

Disclosure: I am the author of the cli-test-helpers Python package.