cPickle:SystemError: error return without exception set

随声附和 提交于 2019-12-07 03:10:22

问题


While dumping large matrix (170000*20000) as follows

 cPickle.dump(train_set,gzip.open('train.pickle.gz','wb'), cPickle.HIGHEST_PROTOCOL)

I get following error:

SystemError: error return without exception set

How would I deal in this case?


回答1:


cPickle can't be used for storing very large objects ( see http://bugs.python.org/issue11564 ).

You have several options:

  • split the data into chunks and store it in multiple files
  • numpy.save
  • h5py <- my favorite because of its convenient numpy-like interface, plus the data are stored in hdf5 which enables interopability with other software/languages.
  • pytables


来源:https://stackoverflow.com/questions/34681401/cpicklesystemerror-error-return-without-exception-set

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!