NVIDIA NVML Driver/library version mismatch

Question

When I run nvidia-smi I get the following message:

Failed to initialize NVML: Driver/library version mismatch

An hour ago I receiv

Answer 1

Mostly reboot would fix the issue on Ubuntu 18.04.

The “Failed to initialize NVML: Driver/library version mismatch?” error generally means the CUDA Driver is still running an older release that is incompatible with the CUDA toolkit version currently in use. Rebooting the compute nodes will generally resolve this issue.