Release hdf5 disk memory after table or node removal with pytables or pandas

非 Y 不嫁゛ 提交于 2019-12-19 08:09:09

问题


I'm using HDFStore with pandas / pytables.

After removing a table or object, hdf5 file size remains unaffected. It seems this space is reused afterwards when additional objects are added to store, but it can be an issue if large space is wasted.

I have not found any command in pandas nor pytables APIs that might be used to recover hdf5 memory.

Do you know of any mechanism to improve data management in hdf5 files?


回答1:


see here

you need to ptrepack it, which rewrites the file.

ptrepack --chunkshape=auto --propindexes --complevel=9 --complib=blosc in.h5 out.h5

as an example (this will also compress the file).



来源:https://stackoverflow.com/questions/21090243/release-hdf5-disk-memory-after-table-or-node-removal-with-pytables-or-pandas

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!