Changing LD_LIBRARY_PATH at runtime for ctypes

Question

How do you update this environment variable at runtime so that ctypes can load a library wherever? I\'ve tried the following and neither seem to work.

from c         


        

Answer 1

CDLL can be passed a fully qualified path name, so for example I am using the following in one of my scripts where the .so is in the same directory as the python script.

import os
path = os.path.dirname(os.path.realpath(__file__))
dll = CDLL("%s/iface.so"%path)

In your case the following should suffice.

from ctypes import *
lib = CDLL("/home/starlon/Projects/pyCFA635/lib/libevaluator.so")

Answer 2

Compile your binary with a rpath relative to the current working directory like:

gcc -shared -o yourbinary.so yoursource.c otherbinary.so \
    -Wl,-rpath='.',-rpath='./another/relative/rpath' -fpic

Then, you are able to change the working directory in python at runtime with:

import os
os.chdir('/path/to/your/binaries')

Like this, the loader also finds other dynamic libraries like otherbinary.so

Answer 3

By the time a program such as Python is running, the dynamic loader (ld.so.1 or something similar) has already read LD_LIBRARY_PATH and won't notice any changes thereafter. So, unless the Python software itself evaluates LD_LIBRARY_PATH and uses it to build the possible path name of the library for dlopen() or an equivalent function to use, setting the variable in the script will have no effect.

Given that you say it doesn't work, it seems plausible to suppose that Python does not build and try all the possible library names; it probably relies on LD_LIBRARY_PATH alone.

Answer 4

Even if you give a fully qualified path to CDLL or cdll.LoadLibrary(), you may still need to set LD_LIBRARY_PATH before invoking Python. If the shared library you load explicitly refers to another shared library and no "rpath" is set in the .so for that library, then it won't be found, even if it has already been loaded. An rpath in a library specifies a search path to be used to search for other libraries needed by that library

For example, I have a case of a set of interdependent third-party libraries not produced by me. b.so references a.so. Even if I load a.so in advance:

ctypes.cdll.LoadLibrary('/abs/path/to/a.so')
ctypes.cdll.LoadLibrary('/abs/path/to/b.so')

I get an error on the second load, because b.so refers to simply 'a.so', without an rpath, and so b.so doesn't know that's the correct a.so. So I have to set LD_LIBRARY_PATH in advance to include '/abs/path/to'.

To avoid having to set LD_LIBRARY_PATH, you modify the rpath entry in the .so files. On Linux, there are two utilities I found that do this: chrpath, and patchelf. chrpath is available from the Ubuntu repositories. It cannot change rpath on .so's that never had one. patchelf is more flexible.

Answer 5

Setting LD_LIBRARY_PATH to the path where libraries are placed won't work here and ctypes will not notice any changes. So, you need to et this at the source and run ldconfig before your script to take this into account. Moreover setting the os environment or any PATH variable in the script will have no effect.

I was facing similar issue and having spent around a day to figure this out.

#mkdir -p /etc/ld.so.conf.d/
#echo "/home/starlon/Projects/pyCFA635/lib" > /etc/ld.so.conf.d/mycustomPath.conf

#ldconfig

Then verify if the path is set by

# ldconfig -v | less

After this is done, try to run your script. This has worked for me and should work for also.

You can see below URL which helped me to resolve this:

https://www.cyberciti.biz/faq/linux-setting-changing-library-path/

Note: I realized the question is old, however wanted to contribute to this as the accepted answer alone was not actually solving the problem.