TypeError: h5py objects cannot be pickled

喜欢而已 提交于 2021-02-11 15:47:32

问题


I am trying to run a PyTorch implementation of a code, which is supposed to work on SBD dataset.

The training labels are originally available in .bin file, which are then converted to HDF5 (.h5) files.

Upon running the algorithm, I get an error as: " TypeError: h5py objects cannot be pickled "

I think the error is stemming from torch.utils.data.DataLoader.

Any idea if I am missing any concept here? I read that pickling is generally not preferred but as of now, my dataset is in HDF5 format only.

For your reference, the error's stack trace is as follows:

  File "G:\My Drive\Debvrat - shared\Codes\CASENet PyTorch Implementations\SBD-lijiaman\main.py", line 130, in <module>
    main()

  File "G:\My Drive\Debvrat - shared\Codes\CASENet PyTorch Implementations\SBD-lijiaman\main.py", line 85, in main
    win_feats5, win_fusion, viz, global_step)

  File "G:\My Drive\Debvrat - shared\Codes\CASENet PyTorch Implementations\SBD-lijiaman\train_val\model_play.py", line 31, in train
    for i, (img, target) in enumerate(train_loader):

  File "C:\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 819, in __iter__
    return _DataLoaderIter(self)

  File "C:\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 560, in __init__
    w.start()

  File "C:\Anaconda3\lib\multiprocessing\process.py", line 112, in start
    self._popen = self._Popen(self)

  File "C:\Anaconda3\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)

  File "C:\Anaconda3\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)

  File "C:\Anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
    reduction.dump(process_obj, to_child)

  File "C:\Anaconda3\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)

  File "C:\Anaconda3\lib\site-packages\h5py\_hl\base.py", line 308, in __getnewargs__
    raise TypeError("h5py objects cannot be pickled")

  TypeError: h5py objects cannot be pickled

I am using Conda version 4.8.2, Python 3.7.4, PyTorch 1.0.0 with Cuda 10.2.89

Thanks,

来源:https://stackoverflow.com/questions/60684061/typeerror-h5py-objects-cannot-be-pickled

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!