LevelDB vs. std::map

Question

In our application we use std::map to store (key, value) data and use serialization to store that data on disk. With this approach we are finding that the disk I/O

Answer 1

What you're doing now is this:

Say you have 1000000 records in a file. You read the whole file into std::map, this takes about ~1000000 operations. You use find/insert to locate and/or insert an element, this takes logarithmic time (about 20 comparisons). And now you save the whole file again, transferring all these 1000000 records back to the file.

The problem is that you benefit absolutely nothing from using std::map. std::map gives you fast search times (logarithmic), but initializing and serializing the whole map per each lookup nullifies it's benefits.

What you need is either redesign you program so you will load the map once at the startup and serialize it once at the termination. Or, perhaps if you need the database semantics, go for a real database implementation. I suggest using SQLite, although LevelDB might be just as good for you.