Which key/value store is the most promising/stable?

Question

I\'m looking to start using a key/value store for some side projects (mostly as a learning experience), but so many have popped up in the recent past that I\'ve got no idea

Answer 1

You need to understand what modern NoSQL phenomenon is about.
It is not about key-value storages. They've been available for decades (BerkeleyDB for example). Why all the fuss now ?

It is not about fancy document or object oriented schemas and overcoming "impedance mismatch". Proponents of these features have been touting them for years and they got nowhere.

It is simply about adressing 3 technical problems: automatic (for maintainers) and transparent (for application developers) failover, sharding and replication. Thus you should ignore any trendy products that do not deliver on this front. These include Redis, MongoDB, CouchDB etc. And concentrate on truly distributed solutions like cassandra, riak etc.

Otherwise you'll loose all the good stuff sql gives you (adhoc queries, Crystal Reports for your boss, third party tools and libraries) and get nothing in return.

Answer 2

Which key value store is the most promising/stable?

G-WAN KV store looks rather promising:

DB engine            Traversal
-----------          ----------------------------
SQLite               0.261 ms  (b-tree)
Tokyo-Cabinet (TC)   4.188 ms  (hash table)
TC-FIXED             0.103 ms  (fixed-size array)
G-WAN KV             0.010 ms  (unamed)

Also, it is used internally by G-WAN webserver, known for its high concurrency performances (that's for the stability question).

Answer 3

As the others said, it depends always on your needs. I for example prefer whatever suits my applications best.

I first used memcached to have fast read/write access. As Java API I´ve used SpyMemcached, what comes with an very easy interface you can use for writing and reading data. Due to memory leaks (no more RAM) I was required to look for another solution, also I was not able scale right, just increase the memory for a single process seemed to be not an good achievement.

After some reviewing I saw couchbase, it comes with replication, clustering, auto-failover, and a community edition (MS Windows, MacOs, Linux). And the best thing for me was, the Java client of it implements also SpyMemcached, so I had almost nothing else to do as setup the server and use couchbase instead of memcached as datastore. Advantage? Sure, my data is now persistent, replicated, and indexed. It comes with a webconsole to write map reduce functions for document views in erlang.

It has Support for Python, Ruby, .Net and more, easy configuration through the webconsole and client-tools. It runs stable. With some tests I was able to write about 10k per second for 200 - 400 byte long records. Reading Performance was way higher though (both tested locally). Have a lot of fun making your decision.

Answer 4

Only have experience with mongoDB, memchache and redis. Here's a comparison between them and couchDB.

Seems mongoDB is most popular. It support sharding and replication, eventually consistent, has good support in ruby (mongoid). It also have a richer feature set than the other two. All of mongo, redis and memchache can store the key-value in memory, but redis seems to be much faster, according to this post, redis is 2x write, 3x read faster than mongo. It has better designed data structures and more 'light-weight'.

I would say they have different usages, mongoDB is probably good for large dataset and document storage while memchache and redis are better to store caches or logs.

Answer 5

At this year's PyCon, Jeremy Edberg of Reddit gave a talk:

http://pycon.blip.tv/file/3257303/

He said that Reddit uses PostGres as a key-value store, presumably with a simple 2-column table; according to his talk it had benchmarked faster than any other key-value store they had tried. And, of course, it's very mature.

Ultimately, OverClocked is right; your use case determines the best store. But RDMBSs have long been (ab)used as key-value stores, and they can be very fast, too.

Answer 6

One distinction you have to make is what will you use the DB for? Don't jump on board just because it's trendy. Do you need a key value store? or do you need a document based store? What is your memory footprint requirement? running it on a small VM or a separate one?

I recommend listing your requirements first and then seeing which ones overlap with your requirements.

With that said, I have used CouchDB/MongoDB and prefer to use MongoDB for its ease of setup and best transition from mysql style queries. I chose mongodb over sql because of dynamic schemas(no migration files!) and better data modeling(arrays, hashes). I did not evaluate based on scalability.

MongoMapper is a great MongoDB orm mapper for Ruby and there's already a working Rails 3 fork.

I listed some more details about why I prefered mongodb in my scribd slides http://tommy.chheng.com/index.php/2010/02/mongodb-for-natural-development/