Tensorflow/nvidia/cuda docker mismatched versions

元气小坏坏 提交于 2020-01-24 21:23:06

问题


I am trying to use tensorflow and nvidia with docker, but hitting the following error:

docker run --runtime=nvidia -it --rm tensorflow/tensorflow:latest-gpu python -c "import tensorflow as tf; tf.enable_eager_execution(); print(tf.reduce_sum(tf.random_normal([1000, 1000])))"

docker: Error response from daemon: OCI runtime create failed: container_linux.go:344: starting container process caused "process_linux.go:424: container init caused \"process_linux.go:407: running prestart hook 1 caused \\"error running hook: exit status 1, stdout: , stderr: exec command: [/usr/bin/nvidia-container-cli --load-kmods configure --ldconfig=@/sbin/ldconfig.real --device=all --compute --utility --require=cuda>=10.0 brand=tesla,driver>=384,driver<385 --pid=5393 /var/lib/docker/overlay2/......./merged]\\nnvidia-container-cli: requirement error: unsatisfied condition: brand = tesla\\n\\"\"": unknown.

I get similar error when trying to run nvidia-smi:

docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi

but when trying to run nvidia-smi with cuda:9.0-base, it works like a charm:

docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi

Do I need to make sure that cuda 10 works or I can run tensorflow with cuda 9? And how can I run the docker image of tensorflow with cuda:9.0-base? (still a docker newby).

Thanks a lot!


回答1:


Ok I think I am finally starting to figure out the mess on my machine.

The tensorflow image does NOT care about the cuda image version, it does not use the docker cuda image. It cares about my nvidia drivers, since it has CUDA integrated in the tensorflow image.

(The docker cuda image that is working with my current drivers, is cuda:9.0)

That meant I have to find a tensorflow image that is working with my drivers (390.116), or update the drivers.

I tried the same command with tensorflow:1.12.0-gpu-py3, and it didn't have any problems.



来源:https://stackoverflow.com/questions/55389669/tensorflow-nvidia-cuda-docker-mismatched-versions

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!