Tokyo Cabinet hash table breaks when reaching 65 GB

点点圈 提交于 2021-02-19 04:29:05

问题


I'm using the Tokyo Cabinet database through the PyTyrant module, storing the data in a hash table (*.tch file). When the file reaches 65 GB in size, the following bug occurs: when a new entry is added to the database, querying the new key gives a "no such key error". However, the new key is listed when I query for a list of all keys. Furthermore, the problematic entries cannot be deleted from the table.

Can anyone explain this?

EDIT: output of 'tcrmgr inform -st -port 1978 localhost' immediately before the error:

version 1.1.40
libver  323
protver 0.91
os  Linux
time    1397633010.989177
pid 25308
sid 0
type    hash
path    tmp.tch
rnum    1104
size    68721564032
bigend  0
fd  8
loadavg 1.630000
memsize 153833472
memrss  2265088
ru_real 3272.949521
cnt_put 1105
cnt_putkeep 0
cnt_putcat  0
cnt_putshl  0
cnt_putnr   0
cnt_out 0
cnt_get 1166
cnt_mget    0
cnt_vsiz    2
cnt_iterinit    5
cnt_iternext    680
cnt_fwmkeys 0
cnt_addint  0
cnt_adddouble   0
cnt_ext 0
cnt_sync    0
cnt_optimize    0
cnt_vanish  0
cnt_copy    0
cnt_restore 0
cnt_setmst  0
cnt_rnum    102
cnt_size    0
cnt_stat    884
cnt_misc    0
cnt_repl    0
cnt_put_miss    0
cnt_out_miss    0
cnt_get_miss    2

and after adding another record, which makes the error appear:

version 1.1.40
libver  323
protver 0.91
os  Linux
time    1397633011.102457
pid 25308
sid 0
type    hash
path    tmp.tch
rnum    1105
size    68731564064
fatal   22: no record found
bigend  0
fd  8
loadavg 1.630000
memsize 153833472
memrss  2265088
ru_real 3273.062801
cnt_put 1106
cnt_putkeep 0
cnt_putcat  0
cnt_putshl  0
cnt_putnr   0
cnt_out 0
cnt_get 1167
cnt_mget    0
cnt_vsiz    2
cnt_iterinit    5
cnt_iternext    680
cnt_fwmkeys 0
cnt_addint  0
cnt_adddouble   0
cnt_ext 0
cnt_sync    0
cnt_optimize    0
cnt_vanish  0
cnt_copy    0
cnt_restore 0
cnt_setmst  0
cnt_rnum    102
cnt_size    0
cnt_stat    885
cnt_misc    0
cnt_repl    0
cnt_put_miss    0
cnt_out_miss    0
cnt_get_miss    3

来源:https://stackoverflow.com/questions/23077952/tokyo-cabinet-hash-table-breaks-when-reaching-65-gb

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!