Use integer keys in Berkeley DB with python (using bsddb3)

戏子无情 提交于 2019-12-12 06:30:33

问题


I want to use BDB as a time-series data store, and planning to use the microseconds since epoch as the key values. I am using BTREE as the data store type.

However, when I try to store integer keys, bsddb3 gives an error saying TypeError: Integer keys only allowed for Recno and Queue DB's.

What is the best workaround? I can store them as strings, but that probably will make it unnecessarily slower.

Given BDB itself can handle any kind of data, why is there a restriction? can I sorta hack the bsddb3 implementation? has anyone used anyother methods?


回答1:


You can't store integers since bsddb doesn't know how to represent integers and which kind of integer it is.

If you convert your integer to a string you will break the lexicographic ordering of keys of bsddb: 10 > 2 but as strings "10" < "2".

You have to use python struct to convert your integers into a string (or in python 3 into bytes) to store then store them in bsddb. You have to use bigendian packing or ordering will not be correct.

Then you can use bsddb's Cursor.set_range(key) to query for information in a given slice of time.

For instance, Cursor.set_range(struct.unpack('>Q', 123456789)) will set the cursor at the key of the even happening at 123456789 or the first that happens after.




回答2:


Well, there's no workaround. But you can use two approaches

  1. Store the integers as string using str or repr. If the ints are big, you can even use string formatting

  2. use cPickle/pickle module to store and retrieve data. This is a good way if you have data types other than basic types. For basics ints and floats this actually is slower and takes more space than just storing strings



来源:https://stackoverflow.com/questions/18664940/use-integer-keys-in-berkeley-db-with-python-using-bsddb3

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!