How to run Tox with Travis-CI

Question

How do you test different Python versions with Tox from within Travis-CI?

I have a tox.ini:

[tox]
envlist = py{27,33,34,35}
recreate = True
         


        

Answer 1

For more control and flexibility you can manually define your matrix so that the Python version and tox environment match up:

language: python
matrix:
  include:
    - python: 2.7
      env: TOXENV=py27
    - python: 3.3
      env: TOXENV=py33
    - python: 3.4
      env: TOXENV=py34
    - python: 3.5
      env: TOXENV=py35
    - python: pypy
      env: TOXENV=pypy
    - env: TOXENV=flake8
install:
  - pip install tox
script:
  - tox

In case it's not obvious, each entry in the matrix starts on a line which begins with a hyphen (-). Any items following that line which are indented are additional lines for that single item.

For example, all entries except for the last, are two lines. the last entry is only one line and does not contain a python setting; therefore, it simply uses the default Python version (Python 2.7 according to the Travis documentation). Of course, a specific Python version is not as important for that test. If you wanted to run such a test against both Python 2 and 3 (once each), then it is recommended to use the versions Travis installs by default (2.7 and 3.4) so that the tests complete more quickly as they don't need to install a non-standard Python version first. For example:

- python: 2.7
  env: TOXENV=flake8
- python: 3.4
  env: TOXENV=flake8

The same works with pypy (second to last entry on matrix) and pypy3 (not shown) in addition to Python versions 2.5-3.6.

While the various other answers provide shortcuts which give you this result in the end, sometimes its helpful to define the matrix manually. Then you can define specific things for individual environments within the matrix. For example, you can define dependencies for only a single environment and avoid the wasted time installing that dependency in every environment.

- python: 3.5
  env: TOXENV=py35
- env: TOXENV=checkspelling
  before_install: install_spellchecker.sh
- env: TOXENV=flake8

In the above matrix, the install_spellchecker.sh script is only run for the relevant environment, but not the others. The before_install setting was used (rather than install), as using the install setting would have overridden the global install setting. However, if that's what you want (to override/replace a global setting), simply redefine it in the matrix entry. No doubt, various other settings could be defined for individual environments within the matrix as well.

Manually defining the matrix can provide a lot of flexibility. However, if you don't need the added flexibility, one of the various shortcuts in the other answers will keep your config file simpler and easier to read and edit later on.

Answer 2

For this I would consider using tox-travis. This is a plugin which allows use of Travis CI’s multiple python versions and Tox’s full configurability. To do this you will configure the .travis.yml file to test with Python:

sudo: false
language: python
python:
    - "2.7"
    - "3.4"
install: pip install tox-travis
script: tox

This will run the appropriate testenvs, which are any declared env with py27 or py34 as factors of the name by default. Py27 or py34 will be used as fallback if no environments match the given factor.

Further Reading

Answer 3

Travis provides the python version for each test as TRAVIS_PYTHON_VERSION, but in the form '3.4', while tox expects 'py34'.

If you don't want to rely on an external lib (tox-travis) to do the translation, you can do that manually:

language: python
python:
-   "2.7"
-   "3.3"
-   "3.4"
-   "3.5"
install:
-   pip install tox
script:
-   tox -e $(echo py$TRAVIS_PYTHON_VERSION | tr -d .)

Search this pattern in a search engine and you'll find many projects using it.

This works for pypy as well:

tox -e $(echo py$TRAVIS_PYTHON_VERSION | tr -d . | sed -e 's/pypypy/pypy/')

Source: flask-mongoengine's .travis.yml.

Answer 4

TOXENV environment variable can be used to select subset of tests for each version of Python via specified matrix:

language: python
python:
  - "2.7"
  - "3.4"
  - "3.5"
env:
  matrix:
    - TOXENV=py27-django-19
    - TOXENV=py27-django-110
    - TOXENV=py27-django-111
    - TOXENV=py34-django-19
    - TOXENV=py34-django-110
    - TOXENV=py34-django-111
    - TOXENV=py35-django-19
    - TOXENV=py35-django-110
    - TOXENV=py35-django-111
install:
  - pip install tox
script:
  - tox -e $TOXENV

In tox config specify to skip missing versions of Python:

[tox]
skip_missing_interpreters=true