Unpickling a python 2 object with python 3

冷暖自知 提交于 2019-11-26 02:52:35

问题


I\'m wondering if there is a way to load an object that was pickled in Python 2.4, with Python 3.4.

I\'ve been running 2to3 on a large amount of company legacy code to get it up to date.

Having done this, when running the file I get the following error:

  File \"H:\\fixers - 3.4\\addressfixer - 3.4\\trunk\\lib\\address\\address_generic.py\"
, line 382, in read_ref_files
    d = pickle.load(open(mshelffile, \'rb\'))
UnicodeDecodeError: \'ascii\' codec can\'t decode byte 0xe2 in position 1: ordinal
not in range(128)

Looking at the pickled object in contention, it\'s a dict in a dict, containing keys and values of type str.

So my question is: Is there a way to load an object, originally pickled in python 2.4, with python 3.4?


回答1:


You'll have to tell pickle.load() how to convert Python bytestring data to Python 3 strings, or you can tell pickle to leave them as bytes.

The default is to try and decode all string data as ASCII, and that decoding fails. See the pickle.load() documentation:

Optional keyword arguments are fix_imports, encoding and errors, which are used to control compatibility support for pickle stream generated by Python 2. If fix_imports is true, pickle will try to map the old Python 2 names to the new names used in Python 3. The encoding and errors tell pickle how to decode 8-bit string instances pickled by Python 2; these default to ‘ASCII’ and ‘strict’, respectively. The encoding can be ‘bytes’ to read these 8-bit string instances as bytes objects.

Setting the encoding to latin1 allows you to import the data directly:

with open(mshelffile, 'rb') as f:
    d = pickle.load(f, encoding='latin1') 

but you'll need to verify that none of your strings are decoded using the wrong codec; Latin-1 works for any input as it maps the byte values 0-255 to the first 256 Unicode codepoints directly.

The alternative would be to load the data with encoding='bytes', and decode all bytes keys and values afterwards.

Note that up to Python versions before 3.6.8, 3.7.2 and 3.8.0, unpickling of Python 2 datetime object data is broken unless you use encoding='bytes'.




回答2:


Using encoding = 'latin1' causes some issues when your object contains numpy arrays in it.

Using encoding = bytes will be better.

Please see this answer for complete explanation of using encoding=bytes



来源:https://stackoverflow.com/questions/28218466/unpickling-a-python-2-object-with-python-3

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!