Can integer keys / values be stored in LevelDB?

Question

I have searched for key value stores that support integer keys and integer values. LevelDB seems a good option, though I can't find any information on whether integer values/keys are supported

Answer 1

You can store pretty much anything in LevelDB. You provide opaque slices of data into LevelDB via the Slice structure. Here is an example:

int intKey = 256;
int intValue = 256*256;

Slice key((char*)&intKey, sizeof(int));
Slice value((char*)&intValue, sizeof(int));

db->Put(leveldb::WriteOptions(), key, value);

And that's pretty much it!

However, one thing to note is that while it's generally fine to store integers in LevelDB (as both keys and values), they will be order via the BytewiseComparator so your key has to support bytewise comparison. This also means that if you rely on specific ordering of the keys, then you have to be mindful of the endian-ness on the system.

You can also write your own comparator via the Comparator interface which will allow you to replace the default BytewiseComparator.

Answer 2

In many cases a more elaborate encoding scheme for integer keys is a better choice. Packing an int into its two-complement representation in a char* (as suggested in another answer to this question) is one option; varint encoding is another one (saves space for small integers, can store arbitrary numbers without an upper bound).

Answer 3

LMDB has explicit support for integer keys (and values, if you're using sorted duplicates). http://symas.com/mdb

When a DB is configured for integer keys, the key comparison functions are also much faster since they can compare word-at-a-time instead of just byte-at-a-time as the default string-oriented compare does.

Disclaimer: I am the author of LMDB. Of course, that doesn't make the facts any different.

Answer 4

To enlarge on Link's answer, partly because I've just been playing with this exact thing as part of the book I'm writing, you can see the BytewiseComparator results he/she talks about below.

Another approach is to flip your binary integers to big endian format so they will sort OK with the default comparator. This makes it easier to compose keys. long flippedI = htonl(i);

Note that LevelDB is very fast. I've done tests on an iPhone4 with 50,000 text-keyed records with secondary keys, so about 100,000 key/value pairs and it screams along.

It is very easy to write a custom Comparator which is used by your database forevermore and still uses ByteWiseComparator for keys other than your numbers. The biggest issue is deciding which keys are covered by your custom rules or not.

A trivial way would be to say that all non-integer keys are more than 4 characters long so you assume a 4 byte key is an integer. That means you just need to ensure you add trailing spaces or something else to push that. It's all very arbitrary and up to you but remember the only two pieces of information you have are the key content and its length. There's no other metadata for a given key.

Part of the results from a sample for standard comparator with int keys starting at 1 and going up by 1 to 1000, using a database with standard BytewiseComparator

Listing the keys in decimal and hex
 256 ( 100)
 512 ( 200)
 768 ( 300)
   1 (   1)
 257 ( 101)
 513 ( 201)
 769 ( 301)
   2 (   2)
 258 ( 102)
 514 ( 202)
 770 ( 302)
   3 (   3)
 259 ( 103)
 515 ( 203)
 771 ( 303)
...
 254 (  fe)
 510 ( 1fe)
 766 ( 2fe)
 255 (  ff)
 511 ( 1ff)
 767 ( 2ff)