Which key/value store is the most promising/stable?

Question

I\'m looking to start using a key/value store for some side projects (mostly as a learning experience), but so many have popped up in the recent past that I\'ve got no idea

Answer 1

Cassandra seems to be popular.

Cassandra is in use at Digg, Facebook, Twitter, Reddit, Rackspace, Cloudkick, Cisco, SimpleGeo, Ooyala, OpenX, and more companies that have large, active data sets. The largest production cluster has over 100 TB of data in over 150 machines.

Answer 2

Which do you recommend, and why?

I recommend Redis. Why? Continue reading!!

Which one is the fastest?

I can't say whether it's the fastest. But Redis is fast. It's fast because it holds all the data in RAM. Recently, virtual memory feature was added but still all the keys stay in main memory with only rarely used values being swapped to disk.

Which one is the most stable?

Again, since I have no direct experience with the other key-value stores I can't compare. However, Redis is being used in production by many web applications like GitHub and Instagram, among many others.

Which one is the easiest to set up and install?

Redis is fairly easy to setup. Grab the source and on a Linux box run make install. This yields redis-server binary that you could put it on your path and start it.

redis-server binds to port 6379 by default. Have a look at redis.conf that comes with the source for more configuration and setup options.

Which ones have bindings for Python and/or Ruby?

Redis has excellent Ruby and Python support.

In response to Xorlev's comment below: Memcached is just a simple key-value store. Redis supports complex data types like lists, sets and sorted sets and at the same time provides a simple interface to these data types.

There is also make 32bit that makes all pointers only 32-bits in size even on 64 bit machines. This saves considerable memory on machines with less than 4GB of RAM.

Answer 3

I really like memcached personally.

I use it on a couple of my sites and it's simple, fast, and easy. It really was just incredibly simple to use, the API is easy to use. It doesn't store anything on disk, thus the name memcached, so it's out if you're looking for a persistent storage engine.

Python has python-memcached.

I haven't used the Ruby client, but a quick Google search reveals RMemCache

If you just need a caching engine, memcached is the way to go. It's developed, it's stable, and it's bleedin' fast. There's a reason LiveJournal made it and Facebook develops it. It's in use at some of the largest sites out there to great effect. It scales extremely well.

Answer 4

There is also zodb.

Answer 5

Just to make the list complete: there's Dreamcache, too. It's compatible with Memcached (in terms of protocol, so you can use any client library written for Memcached), it's just faster.

Answer 6

They all have different features. And don't forget Project Voldemort which is actually used/tested by LinkedIn in their production before each release.

It's hard to compare. You have to ask yourself what you need: e.g. do you want partitioning? if so then some of them, like CouchDB, won't support it. Do you want erasure coding? Then most of them don't have that. Etc.

Berkeley DB is a very basic, low level storage engine, that perhaps can be excused from this discussion. Several key-value systems are built on top of it, to provide additional features like replication, versioning, coding, etc.

Also, what does your application need? Several of the solutions contain complexity that may not be necessary. E.g. if you just store static data that won't change, you can store them under data's SHA-1 content hash (i.e. use the content-hash as key). In this case, you don't have to worry about freshness, synchronization, versioning, and lots of complexities can be removed.