What can I do against 'CUDA driver version is insufficient for CUDA runtime version'?

问题

When I go to /usr/local/cuda/samples/1_Utilities/deviceQuery and execute

moose@pc09 /usr/local/cuda/samples/1_Utilities/deviceQuery $ sudo make clean
rm -f deviceQuery deviceQuery.o
rm -rf ../../bin/x86_64/linux/release/deviceQuery

moose@pc09 /usr/local/cuda/samples/1_Utilities/deviceQuery $ sudo make
"/usr/local/cuda-7.0"/bin/nvcc -ccbin g++ -I../../common/inc  -m64    -gencode arch=compute_20,code=sm_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_37,code=sm_37 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_52,code=compute_52 -o deviceQuery.o -c deviceQuery.cpp
"/usr/local/cuda-7.0"/bin/nvcc -ccbin g++   -m64      -gencode arch=compute_20,code=sm_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_37,code=sm_37 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_52,code=compute_52 -o deviceQuery deviceQuery.o 
mkdir -p ../../bin/x86_64/linux/release
cp deviceQuery ../../bin/x86_64/linux/release

moose@pc09 /usr/local/cuda/samples/1_Utilities/deviceQuery $ ./deviceQuery

I keep getting

./deviceQuery Starting...

CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 35 -> CUDA driver version is insufficient for CUDA runtime version Result = FAIL

I have no idea how to fix it.

My System

moose@pc09 ~ $ cat /etc/issue
Linux Mint 17 Qiana \n \l

moose@pc09 ~ $ uname -a
Linux pc09 3.13.0-36-generic #63-Ubuntu SMP Wed Sep 3 21:30:07 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

moose@pc09 ~ $ lspci -v | grep -i nvidia
01:00.0 VGA compatible controller: NVIDIA Corporation GK110B [GeForce GTX Titan Black] (rev a1) (prog-if 00 [VGA controller])
    Subsystem: NVIDIA Corporation Device 1066
    Kernel driver in use: nvidia
01:00.1 Audio device: NVIDIA Corporation GK110 HDMI Audio (rev a1)
    Subsystem: NVIDIA Corporation Device 1066

moose@pc09 ~ $ sudo lshw -c video
  *-display               
       description: VGA compatible controller
       product: GK110B [GeForce GTX Titan Black]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:01:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
       configuration: driver=nvidia latency=0
       resources: irq:96 memory:fa000000-faffffff memory:d0000000-d7ffffff memory:d8000000-d9ffffff ioport:e000(size=128) memory:fb000000-fb07ffff



moose@pc09 ~ $ nvidia-settings -q NvidiaDriverVersion

  Attribute 'NvidiaDriverVersion' (pc09:0.0): 331.79

moose@pc09 ~ $ cat /proc/driver/nvidia/version 
NVRM version: NVIDIA UNIX x86_64 Kernel Module  331.79  Sun May 18 03:55:59 PDT 2014
GCC version:  gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04) 
moose@pc09 ~ $ lsmod | grep -i nvidia
nvidia_uvm             34855  0 
nvidia              10703828  40 nvidia_uvm
drm                   303102  5 ttm,drm_kms_helper,nvidia,nouveau

moose@pc09 ~ $ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2015 NVIDIA Corporation
Built on Mon_Feb_16_22:59:02_CST_2015
Cuda compilation tools, release 7.0, V7.0.27

moose@pc09 ~ $ nvidia-smi
Thu Nov 12 11:23:24 2015       
+------------------------------------------------------+                       
| NVIDIA-SMI 331.79     Driver Version: 331.79         |                       
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX TIT...  Off  | 0000:01:00.0     N/A |                  N/A |
| 26%   35C  N/A     N/A /  N/A |    132MiB /  6143MiB |     N/A      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Compute processes:                                               GPU Memory |
|  GPU       PID  Process name                                     Usage      |
|=============================================================================|
|    0            Not Supported                                               |
+-----------------------------------------------------------------------------+

回答1:

Update your NVIDIA driver. At the moment you have the driver which only supports CUDA 6 or lower, and you are trying to use the CUDA 7.0 toolkit with it.

回答2:

I ran into this exact same error message with toolkit 8.0 on ubuntu 1604. I tried reinstalling toolkit, cudnn, etc etc and it didn't help. The solution turned out to be very simple: update to the latest NVIDIA driver. I installed NVIDIA-Linux-x86_64-367.57.run and the error went away.

回答3:

For documentation:

If your nvidia version seems appropriate (see: nvcc -V), try and simply use the following before getting into anything messy:

sudo ldconfig

回答4:

My cent,

this error may be related to the selected GPU mode (Performance/Power Saving Mode), when you select (with nvidia-settings utiliy) the integrated Intel GPU and you execute the deviceQuery script... you get this error:

-> CUDA driver version is insufficient for CUDA runtime version

But this error is misleading, by selecting back the NVIDIA GPU(Performance mode) with nvidia-settings utility the problem disappears.

It is not a version problem (in my scenario).

Regards

来源：https://stackoverflow.com/questions/33715746/theano-cuda-is-installed-but-device-gpu-is-not-available

标签

cuda