Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so

Question

I am running a python script and I get this error:

Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so.

Both files are present

Answer 1

Wanted to add on Valilutzik and Zaikun Xu's thread. I can't add comments since I don't have enough points yet.
conda install nomkl numpy scipy scikit-learn numexpr worked for me without having to remove mkl and mkl-service.

Adding an answer to Lee's question if nomkl will be slower: mkl is a Intel math kernel library and is hand-optimized for intel cpus. nomkl uses OpenBlas according to this: https://docs.continuum.io/mkl-optimizations/ It seems that mkl is quite faster on many matrix operations for intel cpus (see https://software.intel.com/en-us/articles/performance-comparison-of-openblas-and-intel-math-kernel-library-in-r)
I've seen someone saying nomkl is faster for AMD cpus (maybe because mkl doesn't work correctly in AMD?)

Answer 2

Just wanted to note that Anaconda 4.0.0, shipped with mkl enabled by default, has this issue. The problem is indeed with Anaconda, as it can be reproduced with the simple python test below.

The actual issue is that Anaconda linked with mkl, but not with libmkl_core.so, thus it has a missing symbol, and can be seen by running:

$ LD_DEBUG=symbols python -c 'import sklearn.linear_model.tests.test_randomized_l1' 2>&1 | grep -i error
      2200:     /opt/anaconda/lib/python2.7/site-packages/scipy/special/../../../../libmkl_avx.so: error: symbol lookup error: undefined symbol: mkl_dft_fft_fix_twiddle_table_32f (fatal)

I didn't want to uninstall mkl, as I'd like to have the performance boost, so I found a workaround which worked for me - preload libmkl_core.so before execution.

$ python -c 'import sklearn.linear_model.tests.test_randomized_l1'
Intel MKL FATAL ERROR: Cannot load libmkl_avx.so or libmkl_def.so.
$
$ LD_PRELOAD=/opt/anaconda/lib/libmkl_core.so python -c 'import sklearn.linear_model.tests.test_randomized_l1'
$

Answer 3

Got the same problem and solved by using the command:

conda install nomkl

I got the solution from this discussion https://github.com/BVLC/caffe/issues/3884

Answer 4

I had this same issue using scikit-learn 0.19 and numpy 1.13.3 when running MLPRegressor (and also with a package called pyearth running an algorithm called MARS). I believe the root of the problem was that our python is part of an Anaconda install, but scikit-learn and numpy were installed via pip, and their expectations for mkl must not agree.

Unfortunately my framework is managed by some dedicated company admins, not by me, so I haven't gotten my guy to try recompiling numpy yet. But I was able to find a workaround based on this thread: Adding export LD_PRELOAD=/path/to/anaconda/lib/libmkl_def.so:/path/to/anaconda/lib/libmkl_avx.so:/path/to/anaconda/lib/libmkl_core.so:/path/to/anaconda/lib/libmkl_intel_lp64.so:/path/to/anaconda/lib/libmkl_intel_thread.so:/path/to/anaconda/lib/libiomp5.so to my ~/.bashrc causes the problem to disappear. It's super hacky, and I'd be lying if I said I knew exactly what it's doing (but this is helpful), so I'm hoping a recompile of numpy is a cleaner fix. But at least it works.

Note it is better performance-wise to have the versions of these packages that use mkl. Installing the nomkl versions is a workaround but not a true solution.

Answer 5

All the solutions provided above did not work for me, but I found a good compromise:

for someone who has the same error and want's to keep Anaconda on his computer, and steel use mkl for (numpy and scipy) good process performance, the solution I propose:

Edit your .bashrc file.

Look for something like export PATH="/home/anaconda2/bin:$PATH" in the file.

Put a # at the beginning to comment it from the script: #export PATH="/home/anaconda2/bin:$PATH"

Open a new terminal and you should be running the base python installation.

It worked for me, hope it will be helpfull.

Answer 6

try export LD_DEBUG=libs and execute your program again.

It will tell you which .so is causing the problem.

Then you can do something, e.g. upgrading the package.

For me, I find upgrade numexpr useful as it prints the following message just before the error:

calling init: xxxxx/python3.6/site-packages/numexpr/interpreter.cpython-36m-x86_64-linux-gnu.so