Issue packaging scrapy spider with cx_Freeze or py2exe

问题

I've created a scraper with Scrapy and wxPython which works as expected, exporting a file with results to the desktop in CSV format. I'm attempting to package this into an executable with cx_Freeze using the below command prompt line:

cxfreeze ItemStatusChecker.py --target-dir dist

This seems to work fine, building the dist directory with ItemStatusChecker.exe

However, when I open ItemStatusChecker.exe, I get the below error in the command prompt and my GUI does not launch:

Traceback (most recent call last):
    File "C:\Python27\lib\site-packages\cx_Freeze\initscripts\Console.py", line 27, in <module>
      exec code in m.__dict__
    File "ItemStatusChecker.py", line 6, in <module>
    File "C:\Python27\lib\site-packages\scrapy\__init__.py", line 6, in <module>
      __version__ = pkgutil.get_data(__package__, 'VERSION').strip()
    File "C:\Python27\lib\pkgutil.py", line 591, in get_data
      return loader.get_data(resource_name_)
IOError: [Errno 2] No such file or directory: 'scrapy\\VERSION'

I've tried running it through py2exe as well. This also seems to work fine, creating the dist directory, but I get a very similar error when trying to launch the exe:

Traceback (most recent call last):
  File "ItemStatusChecker.py", line 6, in <module>
  File "scrapy\__init__.pyc", line 6, in <module>
  File "pkgutil.pyc", line 591, in get_data
IOError: [Errno 2] No such file or directory: 'scrapy\\VERSION'

I'm a python newbie, so let me know if I left out any necessary details. Thank you in advance for any advice you can offer!

回答1:

It sounds like cxfreeze and py2exe aren't picking up the scrapy package automatically. For cx_freeze, you'll probably need to take look at the --include-modules directive:

http://cx-freeze.readthedocs.org/en/latest/script.html#cmdoption--include-modules

For py2exe, you will want to create a setup.py script that has an includes list specified. Here's a tutorial that shows how to do it:

http://www.blog.pythonlibrary.org/2010/07/31/a-py2exe-tutorial-build-a-binary-series/

回答2:

Take a look to (http://cx-freeze.readthedocs.org/en/latest/faq.html#using-data-files)

My solution using cx-freeze:

Modify the file C:\Python2.7\Lib\site-packages\scrapy__init__.py

import sys
# Scrapy version
import pkgutil
if getattr(sys, 'frozen', False):
    __version__ = 'VERSION'.decode('ascii').strip()
else:
    __version__ = pkgutil.get_data(__package__, 'VERSION').decode('ascii').strip()

and the file C:\Python2.7\Lib\site-packages\scrapy\responsetypes.py

import sys
from mimetypes import MimeTypes
from pkgutil import get_data
...
   def __init__(self):
        self.classes = {}
        self.mimetypes = MimeTypes()
        if getattr(sys, 'frozen', False):
            mimedata = 'mime.types'
        else:
            mimedata = get_data('scrapy', 'mime.types')
        self.mimetypes.readfp(StringIO(mimedata))

And in setup.py include this build exe option

"include_files": [("C:\\Python27\\Lib\\site-packages\\scrapy\\VERSION","VERSION"),
    ("C:\\Python27\\Lib\\site-packages\\scrapy\\mime.types","mime.types")]}

来源：https://stackoverflow.com/questions/23351565/issue-packaging-scrapy-spider-with-cx-freeze-or-py2exe

标签

python

wxpython

scrapy

py2exe

cx-freeze