Sqlite load_extension fail for spatialite in Python

Question

I am trying to use the Spatialite beta version 3.0 because I am using Windows 7 on a 64-bit machine.

I consistently get the dreaded sqlite3.OperationalError: The specified module could not be found. error when I try to load libspatialite-4.dll.

I have tried the following:

• put libspatialite-4.dll and all the other dlls in the same folder
• use the full path to the dlls
• add the dll location to the 'PATH' environment variable
• append the dll location to the sys.path attribute as part of the Python code
• copy all the dlls in the c:\windows\system32 folder (complete with restart of the machine)
• copy all the dlls in the c:\windows\sysWoW64 folder (complete with restart of the machine this is supposed to be for 32 bit dlls but I tried it anyway)

my code is as follows:

import sqlite3
conn = sqlite3.connect(":memory:")
conn.enable_load_extension(True)
conn.execute('SELECT load_extension("libspatialite-4.dll")')

NOTE - I have tried the full path too with no luck. I remember having the same problem with Windows XP 32-bit. I got it working but can't remember what I did :(

UPDATE

I have tested the setup on 32-bit Windows 7 and putting all the dlls in the System32 folder works. So, this suggests that there is some problem with the 64-bit setup. Could it be that I need another version of MSVC (I don't think the Spatialite website says which is necessary so I might just have to guess - I have MSVC2010 installed)?

Answer 1

I was having the same problem and it confused me for days. I'm running Python 2.6 (32-bit) on Windows 7(x64), so it may not be the same setup as you have.

Here's what you can try (taken from this Google Groups post):

1. Put the spatialite dlls (libgeos_c-1.dll, libgeos-3-0-0.dll, libproj-0.dll, and libspatialite-2.dll) into a folder on the C:/ drive.
2. Add this folder to your PATH.

The problems seems to be that Windows may not load the dlls from C:\Windows\system32 because of user permissions settings.

Answer 2

I just recently went through a nightmare of a x64 build of pyspatialite 3.0.1 and all its dependent libraries. It can be done, but takes some "tweaking" to get it right.

First, note the workaround that may be needed to compile spatialite.c amalgamation here:

Pyspatialite 3.0.1 Issue #7 Comment #3

Second, I suggest you compile with MSVC 2008 / SDK 7.0 x64, which is what Python 2.7 x64 is compiled with. I ran into a great deal of trouble trying to get things to compile right when I compiled the dependency libs with mingw-w64.

iconv (v. 1.9.2) and proj4 (v. 4.8.0) both seem to compile and install just fine; however, you may run into some trouble using the latest GEOS svn_trunk (v. 3.3.5). Note the following links for workarounds/fixes...

OSGEO GEOS TRAC Ticket #574

OSGEO GEOS TRAC Ticket #577

If you download the two makefiles from 577, they include the fix in 574.

You will also want to download one of the nightly snapshots and copy the geos/src/triangulate directory over to your build folder before compiling, as it's missing in the svn_trunk.

Finally, you'll want to make a minor modification to geos/src/dirlist.mk:

At line 45, add 'triangulate \' (no quotes) just below 'simplify \' and above 'util'.

Now when you compile, you may see some warnings, but the build should not fail outright...

nmake /f makefile.vc PREFIX=../Path/To/Geos/Install/Here

nmake install /f makefile.vc PREFIX=../Path/To/Geos/Install/Here

That takes care of your lib dependencies. Now you need to do one of two things: 1.Either create a setup.cfg file in your pyspatialite build folder and add the /bin, /lib, and /include paths, or 2.Directly edit the pyspatialite setup.py file and do the same.

I found it easiest to edit the setup.py file directly, and add the paths to the dependency libs to look similar to:

(line 45) include_dirs = ['../usr/local/include', '../python27/include']

(line 46) library_dirs = ['../usr/local/lib', '../python27/libs', '../usr/local/bin', '../python27/DLLs']

(line 47) libraries = ['geos','geos_c','proj','iconv'] # You may need to add 'iconv' here

(line 48) runtime_library_dirs = ['../usr/local/lib', '../python27/libs', '../usr/local/bin', '../python27/DLLs']

If after making these changes pyspatialite still fails to build for you, then make one more set of modifications to setup.py: around line 121, add the following lines...

ext.include_dirs.append('../python27/include') 
ext.include_dirs.append('../usr/local/include') 

ext.library_dirs.append('../python27/libs')
ext.library_dirs.append('../usr/local/lib')

ext.library_dirs.append('../python27/DLLs')
ext.library_dirs.append('../usr/local/bin')

Remember to replace the paths to match your particular setup. That should do it. After you run 'python setup.py install' everything should work.

You can run all the tests in ../Python27/Lib/site-packages/pyspatialite/test - they all passed for me; however, a better, more realistic test might be to run the example code from this link:

SpatiaLite and Python

The steps the author goes into don't cover the detail to get the dependency libs working in an x64 bit environment, however, and I didn't find them particularly useful as pyspatialite 3.0.1 now automatically detects the appropriate version of the spatialite amalgamation to download. The sample code on the site creates a spatialite database file and populates it with thousands of entries. Everything ran successfully for me; so I believe the method outlined above to get a pyspatialite x64 build works.

Good luck!

-RMWChaos

Answer 3

The version of sqlite3.dll included with Python doesn't seem to want to play nice with Spatialite. The only thing I could get to work (short of compiling everything from source) was:

1. Download SQLite (or cyqlite - a recompile of SQLite for Windows with some handy features enabled, such as R-Tree so you can do spaital indexes) i.e. sqlite-dll-win32-x86-[version].zip
2. Download mod_spatialite (Windows binaries are in the pink box at the bottom of the page) i.e. mod_spatialite-[version]-win-x86.7z
3. Unzip first SQLite/cyqlite then mod_spatialite into the same folder (overwrite if there are any conflicts)
4. Add this folder to your system Path
5. Rename the sqlite3.dll that is in your Python DLLs directory, to something like sqlite3_old.dll, so that Python will use the new one on your path

See this blog post for more info.

Answer 4

I solved this by following instructions in the last post here. My system was, as per some above - Win7 64bit with 32-bit spatialite libraries and the pysqlite library for python. I linked to the sqlite3.dll in my DLLs directory (this is the Python that ArcGIS installs, so slightly different to other installations).

This solved the problem with loadinging Spatialite 4 dll via pysqlite.