Is there a “multi-user” Docker mode, e.g. for scientific clusters?

Question

I want to use Docker for isolating scientific applications for the use in a HPC Unix cluster. Scientific software often has exotic dependencies so isolating them with Docker app

Answer 1

Don't forget about DinD (Docker in Docker): jpetazzo/dind

You could dedicate one Docker per user, and within one of those docker containers, the user could launch a job in a docker container.

Answer 2

Yes there is! It's called Singularity and it was designed with scientific applications and multi user HPCs. More at http://singularity.lbl.gov/

Answer 3

There is an officially supported Docker image that allows one to run Docker in Docker (dind), available here: https://hub.docker.com/_/docker/. This way, each user can have their own Docker daemon. First, start the daemon instance:

docker run --privileged --name some-docker -d docker:stable-dins

Note that the --privileged flag is required. Next, connect to that instance from a second container:

docker run --rm --link some-docker:docker docker:edge version

Answer 4

OK, I think there will be more and more solutions pop up for this. I'll try to update the following list in the future:

• udocker for executing Docker containers as users
• Singularity (Kudos to Filo) is another Linux container based solution

Answer 5

I'm also interested in this possibility with Docker, for similar reasons. There are a few of problems I can think of:

1. The Docker Daemon runs as root, providing anyone in the docker group with effective host root permissions (e.g. leak permissions by mounting host / dir as root).
2. Multi user Isolation as mentioned
3. Not sure how well this will play with any existing load balancers?

I came across Shifter which may be worth a look an partly solves #1: http://www.nersc.gov/research-and-development/user-defined-images/

Also I know there is discussion to use kernel user namespaces to provide mapping container:root --> host:non-privileged user but I'm not sure if this is happening or not.