I can't load my model because I can't put a PosixPath

问题

I'm setting up a script and I need to use some functions from fast-ai package. The fact is that I'm on Windows and when I define my paths, the function from fast-ai named load_learner can't load the model.

I've tried to change the function into the package as:

state = pickle.load(open(str(path) + '/' + str(fname), 'rb'))

instead of:

state = pickle.load(open(path/fname, 'rb'))

but I obtain this error:

 File "lib\site-packages\fastai\basic_train.py", line 462, in load_learner
    state = pickle.load(open(path/fname, 'rb'))
  File "\lib\pathlib.py", line 1006, in __new__
    % (cls.__name__,))
NotImplementedError: cannot instantiate 'PosixPath' on your system

My paths are defined as:

folder_path = './models/model1'
fname = 'model.pkl'

and I call the function as: model = load_learner(folder_path, fname)

How can I use Windows paths in this function?

---

UPDATE 1

The answer posted was correct only on Linux. I still have the issue on Windows. I didn't find a way to pass through the PosixPath on Windows. The only solution that I found is to change the internal packages from my modules but it is not a secure way to solve this kind of issue.

---

Thanks in advance.

回答1:

According my own question, I find a way using:

from pathlib import Path

folder_path = Path('./models/model1')

---

UPDATE 1

This solution only works on Linux, on Windows I still get an error.

---

回答2:

According to provided error message you are using pathlib. So you don't need to use + '/' + here: str(path) + '/' + str(fname)

/ as path separator works on Linux/Unix:

state = pickle.load(open(path / fname, 'rb'))

On Windows use .joinpath() instead:

state = pickle.load(open(path.joinpath(fname), 'rb'))

If you are not going to use the pathlib, use os.path.join(). It will automatically select right format for your OS.

回答3:

The issue here is related to the differences in the way Python handles paths depending on the OS:

• PosixPath - on Linux / Unix

• WindowsPath - on Windows

When persisting objects using pickle on one OS (say Linux - as in this case) information about the type / class is persisted too (here: PosixPath).

Now, when the pickle file is being loaded Python assumes it is going to be able to recreate objects based on the type information it previously persisted. In this case it tries to recreate an object of PosixPath type which is prevented by pathlib library and cannot be instantiated on Windows. On Windows there should be WindowsPath be used instead but pickle module does not handle such OS-dependent logic very well hence it helplessly throws the error.

You could theoretically interfere in the code of pathlib to remove the OS check but there is no easy workaround but avoiding pickling of OS-dependent objects (e.g. storing paths as strings - as os.path does - would certainly workaround this issue).

There is also another possibility - to use a platform-independent PurePosixPath class for path objects.

回答4:

I am working on the same- deploying fastai model as a web server and got the same issue and this is what I did... While exporting the model use joblib or pickle to pickle the model instead of using learn.export() and in server use the below code.

 __model = pickle.load(open(os.path.join('./artifacts/saved_model.pkl'), 'rb'))

By doing this it is able to solve the path issue but since the model is trained using GPU it gives the error which asks to map the storage to CPU

来源：https://stackoverflow.com/questions/57286486/i-cant-load-my-model-because-i-cant-put-a-posixpath