Is there a way to sandbox test execution with pytest, especially filesystem access?

Question

I\'m interested in executing potentially untrusted tests with pytest in some kind of sandbox, like docker, similarly to what continuous integration services do.

I un

Answer 1

To be honest, this seems like a great use case for something like docker. Sure, you're not handling it completely cleanly using only python, but you can abuse the host OS to your heart's desire and not have to worry about long-term damage. Plus, unlike a lot of CI solutions, it can run comfortably on your development machine.

Also note that whether or not your code is intentionally malicious or not, having that sort of isolation is still beneficial to prevent accidents like:

rm -rf /usr/local/share/ myapp

Answer 2

After quite a bit of research I didn't find any ready-made way for pytest to run a project tests with OS-level isolation and in a disposable environment. Many approaches are possible and have advantages and disadvantages, but most of them have more moving parts that I would feel comfortable with.

The absolute minimal (but opinionated) approach I devised is the following:

• build a python docker image with:
  • a dedicated non-root user: pytest
  • all project dependencies from requirements.txt
  • the project installed in develop mode
• run py.test in a container that mounts the project folder on the host as the home of pytest user

To implement the approach add the following Dockerfile to the top folder of the project you want to test next to the requirements.txt and setup.py files:

FROM python:3

# setup pytest user
RUN adduser --disabled-password --gecos "" --uid 7357 pytest
COPY ./ /home/pytest
WORKDIR /home/pytest

# setup the python and pytest environments
RUN pip install --upgrade pip setuptools pytest
RUN pip install --upgrade -r requirements.txt
RUN python setup.py develop

# setup entry point
USER pytest
ENTRYPOINT ["py.test"]

Build the image once with:

docker build -t pytest .

Run py.test inside the container mounting the project folder as volume on /home/pytest with:

docker run --rm -it -v `pwd`:/home/pytest pytest [USUAL_PYTEST_OPTIONS]

Note that -v mounts the volume as uid 1000 so host files are not writable by the pytest user with uid forced to 7357.

Now you should be able to develop and test your project with OS-level isolation.

Update: If you also run the test on the host you may need to remove the python and pytest caches that are not writable inside the container. On the host run:

rm -rf .cache/ && find . -name __pycache__  | xargs rm -rf