Porting from SQLite to Redis

Question

I was using SQLite for an app in which I used to store 8-10 columns. I used to retrieve the data based upon a combination of any number of those attributes. Now I want to po

Answer 1

I think the best advice is to avoid sticking to the relational model when porting something from a RDBMS to Redis. And beyond the model, an important difference is to focus on data access paths as well as the data structures.

Redis does not include a query language (but rather commands a la memcached), and cannot therefore reply to arbitrary queries. If an access path to the data is not part of the data structure, then the data cannot be efficiently retrieved.

Redis is not the best NoSQL store when it comes to supporting arbitrary queries. For instance, you would be better served by something like MongoDB.

Now, if you really want to implement your stuff with Redis, you could try to use a strategy similar to tagging engines. Your records can be stored in hash objects. For each column part of the arbitrary queries you need to support, you build reverse indexes using sets.

For instance:

# Set up the records: one hash object per record
hmset user:1 name Bilbo type Hobbit job None
hmset user:2 name Frodo type Hobbit job None
hmset user:3 name Gandalf type Maiar job Wizard
hmset user:4 name Aragorn type Human job King
hmset user:5 name Boromir type Human job Warrior

# Set up the indexes: one set per value per field
sadd name:Bilbo 1
sadd name:Frodo 2
sadd name:Gandalf 3
sadd name:Aragorn 4
sadd name:Boromir 5
sadd type:Hobbit 1 2
sadd type:Maiar 3
sadd type:Human 4 5
sadd job:None 1 2
sadd job:Wizard 3
sadd job:King 4
sadd job:Warrior 5

# Perform a query: we want the humans who happen to be a king
# We just have to calculate the intersection of the corresponding sets
sinterstore tmp type:Human job:King
sort tmp by nosort get user:*->name get user:*->job get user:*->type
1) "Aragorn"
2) "King"
3) "Human"

By combining union, intersection, difference, more complex queries can be implemented. For non discrete values, or for range based queries, ordered sets (zset) have to be used (and can be combined with normal sets).

This method is usually quite fast if the values are discriminant enough. Please note you do not have the flexibility of a RDBMS though (no regular expressions, no prefixed search, range queries are a pain to deal with, etc ...)