ubuntu 11.04 lxml import etree problem for custom python

Question

ubuntu 11.04 has native python2.7 i build python2.5 from source to /usr/local/python2.5/bin, and try to install lxml for my custom python2.5 install. Also i use virtualenv.

Answer 1

This problem is usually caused by building Python without using the --enable-unicode=ucs4 option on the ./configure command.

To make sure you do it right, delete the existing Python build directory and start building again by unzipping the Python tarball.

Also, delete the existing Python 2.5 install directory /usr/local/python2.5/ and rebuild everything else that you built such as lxml. Any extensions that use compiled components will look up the Python build configuration so if you don't rebuild everything you will have mismatched pieces.

Answer 2

From this thread I found, they seemed to be stating that the problem might be caused by the program being compiled against the wrong Python version. 2.7 vs 2.5, it might be the cause of the problem for you. Check your sys.path and see if it's using the 2.7 version, and that just might be the source of your problem!

If it's not using the 2.7 one, maybe it was still compiled incorrectly on your system anyways. Maybe it's a packaging error on your distro.

Answer 3

You cannot directly symlink different Python versions or native libraries, as Python DLL format changes across major Python versions.

Based on this:

"from lxml import etree" raise "ImportError: /home/se7en/.virtualenvs/e-py25/lib/python2.5/site-packages/lxml-2.2.4-py2.5-linux-i686.egg/lxml/etree.so: undefined symbol: PyUnicodeUCS2_DecodeLatin1

It clearly states that lxml is somehow getting compiled against wrong version of Python. Usually this error stems from the problem that you have been mixing manually compiled Python interpreter with Ubuntu's default one, as Python interpreter can be compiled with different unicode flags and Ubuntu uses non-default flags (if I recall correctly).

Usually I solve this problem by

• Creating a fresh virtualenv

• Reinstalling lxml under this virtualenv using easy_install

• Running Python using -v switch and Python will print everything it tries to import

• If it is still importing stuff from wrong location either virtualenv or your native library setup has been corrupted

• Native library setup can be overridden with manual lib builds and LD_LIBRARY_PATH environment variable

• If virtualenv does not build lxml against your correct Python version it is virtualenv bug (as long as you can show how to reproduce this in a repeatable manner). However, we have been succesfully using lxml with Ubuntu, virtualenv and various Python versions so I doubt there is a bug.

There is also a way for static lxml installations using a tool called buildout (a little bit like virtualenv, but much much more complex):

http://groups.google.com/group/gomobile-dev/browse_thread/thread/7f5e34e991cfdaa9/c65b70e7a9422ebf?#c65b70e7a9422ebf

Answer 4

try this to install lxml under your virtualenv, hopefully you won't get any errors

/home/se7en/.virtualenvs/e-py25/bin/activate
easy_install pip # if you don't have it already
pip install lxml
python -c 'import lxml' # to confirm all is good

Answer 5

I un-installed the existing lxml (which I installed using sudo apt-get install python-lxml) by using the command sudo apt-get remove python-lxml. And I installed lxml 4.0.0 manually. This solved the problem for me. I think this new version is debugged one.