How would I learn more about sharding userdata for a website?

Question

I\'m interested in sharding my websites user data across multiple servers.

For example, users will login from the same place. but the login script needs to figure out wh

Answer 1

You should check the very informative site http://highscalability.com . Posts worth reading:

• Flickr Architecture and Federation
• Livejournal Architecture
• Habits of Highly Scalable Web Applications

Generally you are following the right approach but this can get nasty quite fast if you need to do queries on more than one cluster - e.g. "your friends' recent posts" type queries.

Answer 2

One option you might want to consider: use a simple hash. For example, take the MD5 hash of the username, then treat the last 8 bytes of that as a long. Take that long mod (number of servers) and make that the server to put the data on. That way you don't need any central registry/configuration other than an ordered list of servers.

The disadvantage is that changing the number of servers involves moving all the data to the new "correct" location...

(There's also the matter that if one machine goes down, those users are stuffed - you'll want to consider having some sort of redundancy.)