Using GPU from a docker container?

Question

I\'m searching for a way to use the GPU from inside a docker container.

The container will execute arbitrary code so i don\'t want to use the privileged mode.

<

Answer 1

Regan's answer is great, but it's a bit out of date, since the correct way to do this is avoid the lxc execution context as Docker has dropped LXC as the default execution context as of docker 0.9.

Instead it's better to tell docker about the nvidia devices via the --device flag, and just use the native execution context rather than lxc.

Environment

These instructions were tested on the following environment:

• Ubuntu 14.04
• CUDA 6.5
• AWS GPU instance.

Install nvidia driver and cuda on your host

See CUDA 6.5 on AWS GPU Instance Running Ubuntu 14.04 to get your host machine setup.

Install Docker

$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
$ sudo sh -c "echo deb https://get.docker.com/ubuntu docker main > /etc/apt/sources.list.d/docker.list"
$ sudo apt-get update && sudo apt-get install lxc-docker

Find your nvidia devices

ls -la /dev | grep nvidia

crw-rw-rw-  1 root root    195,   0 Oct 25 19:37 nvidia0 
crw-rw-rw-  1 root root    195, 255 Oct 25 19:37 nvidiactl
crw-rw-rw-  1 root root    251,   0 Oct 25 19:37 nvidia-uvm

Run Docker container with nvidia driver pre-installed

I've created a docker image that has the cuda drivers pre-installed. The dockerfile is available on dockerhub if you want to know how this image was built.

You'll want to customize this command to match your nvidia devices. Here's what worked for me:

 $ sudo docker run -ti --device /dev/nvidia0:/dev/nvidia0 --device /dev/nvidiactl:/dev/nvidiactl --device /dev/nvidia-uvm:/dev/nvidia-uvm tleyden5iwx/ubuntu-cuda /bin/bash

Verify CUDA is correctly installed

This should be run from inside the docker container you just launched.

Install CUDA samples:

$ cd /opt/nvidia_installers
$ ./cuda-samples-linux-6.5.14-18745345.run -noprompt -cudaprefix=/usr/local/cuda-6.5/

Build deviceQuery sample:

$ cd /usr/local/cuda/samples/1_Utilities/deviceQuery
$ make
$ ./deviceQuery   

If everything worked, you should see the following output:

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 6.5, CUDA Runtime Version = 6.5, NumDevs =    1, Device0 = GRID K520
Result = PASS

Answer 2

Writing an updated answer since most of the already present answers are obsolete as of now.

Versions earlier than Docker 19.03 used to require nvidia-docker2 and the --runtime=nvidia flag.

Since Docker 19.03, you need to install nvidia-container-toolkit package and then use the --gpus all flag.

So, here are the basics,

Package Installation

Install the nvidia-container-toolkit package as per official documentation at Github.

For Redhat based OSes, execute the following set of commands:

$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
$ curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.repo | sudo tee /etc/yum.repos.d/nvidia-docker.repo

$ sudo yum install -y nvidia-container-toolkit
$ sudo systemctl restart docker

For Debian based OSes, execute the following set of commands:

# Add the package repositories
$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
$ curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
$ curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

$ sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
$ sudo systemctl restart docker

Running the docker with GPU support

docker run --name my_all_gpu_container --gpus all -t nvidia/cuda

Please note, the flag --gpus all is used to assign all available gpus to the docker container.

To assign specific gpu to the docker container (in case of multiple GPUs available in your machine)

docker run --name my_first_gpu_container --gpus device=0 nvidia/cuda

Or

docker run --name my_first_gpu_container --gpus '"device=0"' nvidia/cuda

Answer 3

Updated for cuda-8.0 on ubuntu 16.04

• Install docker https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-16-04

• Build the following image that includes the nvidia drivers and the cuda toolkit

Dockerfile

FROM ubuntu:16.04
MAINTAINER Jonathan Kosgei <jonathan@saharacluster.com>

# A docker container with the Nvidia kernel module and CUDA drivers installed

ENV CUDA_RUN https://developer.nvidia.com/compute/cuda/8.0/prod/local_installers/cuda_8.0.44_linux-run

RUN apt-get update && apt-get install -q -y \
  wget \
  module-init-tools \
  build-essential 

RUN cd /opt && \
  wget $CUDA_RUN && \
  chmod +x cuda_8.0.44_linux-run && \
  mkdir nvidia_installers && \
  ./cuda_8.0.44_linux-run -extract=`pwd`/nvidia_installers && \
  cd nvidia_installers && \
  ./NVIDIA-Linux-x86_64-367.48.run -s -N --no-kernel-module

RUN cd /opt/nvidia_installers && \
  ./cuda-linux64-rel-8.0.44-21122537.run -noprompt

# Ensure the CUDA libs and binaries are in the correct environment variables
ENV LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-8.0/lib64
ENV PATH=$PATH:/usr/local/cuda-8.0/bin

RUN cd /opt/nvidia_installers &&\
    ./cuda-samples-linux-8.0.44-21122537.run -noprompt -cudaprefix=/usr/local/cuda-8.0 &&\
    cd /usr/local/cuda/samples/1_Utilities/deviceQuery &&\ 
    make

WORKDIR /usr/local/cuda/samples/1_Utilities/deviceQuery

3. Run your container

sudo docker run -ti --device /dev/nvidia0:/dev/nvidia0 --device /dev/nvidiactl:/dev/nvidiactl --device /dev/nvidia-uvm:/dev/nvidia-uvm <built-image> ./deviceQuery

You should see output similar to:

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 8.0, CUDA Runtime Version = 8.0, NumDevs = 1, Device0 = GRID K520 Result = PASS

Answer 4

To use GPU from docker container, instead of using native Docker, use Nvidia-docker. To install Nvidia docker use following commands

curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey |  sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/ubuntu16.04/amd64/nvidia-
docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
sudo apt-get install -y nvidia-docker
sudo pkill -SIGHUP dockerd # Restart Docker Engine
sudo nvidia-docker run --rm nvidia/cuda nvidia-smi # finally run nvidia-smi in the same container