Package spacy model | 易学教程

问题

I want to include the spacy model de_core_news_sm in a python package.

Here is my project: https://github.com/michaelhochleitner/package_de_core_news_sm .

I package and install the project with the following commands.

python setup.py sdist bdist_wheel
pip install dist/example-pkg-mh-0.0.1.tar.gz

I want to import the module example_pkg.import-model.py .

$ python
>>> import example_pkg.import_model
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mh/PycharmProjects/packaging_tutorial/example_pkg/import_model.py", line 2, in <module>
    import de_core_news_sm
ModuleNotFoundError: No module named 'de_core_news_sm'

How can I include the module 'de_core_news_sm' into the package so that it's installed after running the following command?

pip install dist/example-pkg-mh-0.0.1.tar.gz

回答1:

If you want to spare your users from running that download, you'll have to package and distribute it with your own source. This process is called vendoring (see this great post for an in-depth explanation of how to best do it in python, or the pip-project's _vendor/__init__.py for a commented example), and it can be quite convenient, but prone to get you into annoying issues if overdone.

Simply put, you create an additional python package called _vendor (or something similar) in the source code directory of your package and copy the downloaded de_core_news_sm package into it:

example_pkg
├── import_model.py
├── __init__.py
└── _vendor
    ├── __init__.py
    └── de_core_news_sm
        ├── de_core_news_sm-2.1.0
        │   ├── accuracy.json
        │   ├── meta.json
        │   ├── ner/
        │   ├── parser/
        │   ├── tagger/
        │   ├── vocab/
        │   └── tokenizer
        ├── __init__.py
        └── meta.json

You can find the package in the site_packages of the python interpreter that you installed it into with python -m spacy download de_core_news_sm, i.e. $(which python)/site_packages/de_core_news_sm.

Finally, you need to change all your imports of the model from import de_core_news_sm to from example_pkg._vendor import de_core_news_sm, and then it should work.

回答2:

Did you try on command line:

python -m spacy download de_core_news_sm

来源：https://stackoverflow.com/questions/57773454/package-spacy-model

标签

python

spacy

python-packaging