FastText - Cannot load model.bin due to C++ extension failed to allocate the memory

问题

I'm trying to use the FastText Python API https://pypi.python.org/pypi/fasttext Although, from what I've read, this API can't load the newer .bin model files at https://github.com/facebookresearch/fastText/blob/master/pretrained-vectors.md as suggested in https://github.com/salestock/fastText.py/issues/115

I've tried everything that is suggested at that issue, and furthermore https://github.com/Kyubyong/wordvectors doesn't have the .bin for English, otherwise the problem would be solved. Does anyone know of a work-around for this?

回答1:

Try the following steps to solve the issue. Its worked with me for Python 3.5 while loading a language identification model.

git clone https://github.com/facebookresearch/fastText.git
cd fastText
pip install .

回答2:

You can't use the package fasttext installed by pip. You use install fastText (capital T) from fasttext repo.

回答3:

This error generally comes because of the version mismatch. Make sure you are using the same version of fasttext during the training and model loading time.

回答4:

The package you are looking for is not on PyPI yet. You have to install it yourself by cloning the original repo and running the install script.

There is, however, an up-to-date version in PyPi Test which you can test and use.

You can also use this mirror, which has a copy of the official package you are looking for.

回答5:

you may try pyfasttext instead:

!pip3 install pyfasttext
from pyfasttext import FastText
model = FastText("./model_position")

works, but some compatibility problems, e.g. you may use model.args["dim"] to replace get_dimension()

来源：https://stackoverflow.com/questions/45923279/fasttext-cannot-load-model-bin-due-to-c-extension-failed-to-allocate-the-mem