Something wrong with how I'm bundling rasterio into an executable

问题

Expected behavior and actual behavior.

I expected to compile a script using rasterio into an executable using pyinstaller. The script runs fine from within my python environment. However I am not able to freeze it into an executable using PyInstaller.

Steps to reproduce the problem.

I have a script called workflow_3.py which contains the following:

import rasterio

That's it. I tried to compile then run this using pyinstaller as follows:

(wps_env36) D:\11202750-002_RA2CE\Basis>pyinstaller workflow_3.py (wps_env36) D:\11202750-002_RA2CE\Basis>dist\workflow_3\workflow_3.exe

The compilation seems to run to completion, however when I run the executable I get the following error:

(wps_env36) D:\11202750-002_RA2CE\Basis>dist\workflow_3\workflow_3.exe
Traceback (most recent call last):
  File "workflow_3.py", line 1, in <module>
    import rasterio
  File "c:\programdata\anaconda2\envs\wps_env36\lib\site-packages\PyInstaller\loader\pyimod03_i
mporters.py", line 627, in exec_module
    exec(bytecode, module.__dict__)
  File "site-packages\rasterio\__init__.py", line 23, in <module>
  File "rasterio\_base.pyx", line 1, in init rasterio._base
ModuleNotFoundError: No module named 'rasterio._shim'
[17536] Failed to execute script workflow_3

Attempt to fix the problem

I modified the spec file by explicitly adding 'rasterio._shim' to the list contained by the hidden-imports variable. Then I ran pyinstaller workflow_3.spec . This caused other ModuleNotFoundError for modules such as control.py, crs.py and vrt.py.

Adding these to hidden-imports successfully eliminates the ModuleNotFoundError for that particular package but it still looks for other packages, all of which are contained in C:\ProgramData\Anaconda2\envs\wps_env36\Lib\site-packages\rasterio. There are about 40 modules in this directory. It seems excessive to add every single filename in this directory to the hidden-imports variable. In fact I don't even know if it would work.

Therefore, I also tried adding that whole directory into my pathex variable so that it can extend the PYTHONPATH with it. However this causes another problem:

File "c:\programdata\anaconda2\envs\wps_env36\lib\traceback.py", line 5, in <module> File "c:\programdata\anaconda2\envs\wps_env36\lib\linecache.py", line 11, in <module> File "c:\programdata\anaconda2\envs\wps_env36\lib\tokenize.py", line 27, in <module> ImportError: cannot import name 'open' pre-safe-import-module hook failed, needs fixing.

Operating system

Windows 7

Rasterio version and provenance

The rasterio version is 1.0.8, from conda-forge The python version is 3.6.6

I have two versions of pyinstaller

pyinstaller               3.4              py36h7602738_0    conda-forge
PyInstaller               3.5.dev0+b13e6b30b           <pip>

The second one is the development version, which I had to get because of this problem

Question

How do I use PyInstaller to freeze an application which uses rasterio?

回答1:

The current solution that I came up with is to force feed hidden-imports variable all modules contained within C:\ProgramData\Anaconda2\envs\wps_env36\Lib\sitepackages\rasterio using the glob package. In my spec file I added some python code to do this:

# -*- mode: python -*-

block_cipher = None
import glob, os
rasterio_imports_paths = glob.glob(r'C:\ProgramData\Anaconda2\envs\wps_env36\Lib\site-packages\rasterio\*.py')
rasterio_imports = ['rasterio._shim']

for item in rasterio_imports_paths:
    current_module_filename = os.path.split(item)[-1]
    current_module_filename = 'rasterio.'+current_module_filename.replace('.py', '')
    rasterio_imports.append(current_module_filename)

a = Analysis(['workflow_3.py'],
             pathex=['D:\\11202750-002_RA2CE\\Basis'],
             binaries=[],
             datas=[],
             hiddenimports=rasterio_imports,
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          [],
          exclude_binaries=True,
          name='workflow_3',
          debug=True,
          bootloader_ignore_signals=False,
          strip=False,
          upx=True,
          console=True )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=False,
               upx=True,
               name='workflow_3')

Unfortunately this does not explain why pyinstaller was not able to see those modules in the first place. However it does momentarily solve this problem, and the code compiles fine.

来源：https://stackoverflow.com/questions/53149750/something-wrong-with-how-im-bundling-rasterio-into-an-executable

标签

python

pyinstaller

rasterio