What's the difference between ZooKeeper and any distributed Key-Value stores?

Question

I am new to zookeeper and distributed systems, and am learning it myself.

From what I understand for now, it seems that ZooKeeper is simply a key-value store whose keys

Answer 1

You can compare zookeeper with other distributed key-value store such as etcd and consul. These tools are also offering the same benefits of apache zookeeper. The main advantage of zookeeper is that it takes care of avoiding deadlock and race condition in a distributed applications. Zookeeper is not only a key-value store, It can be also used for service discovery and centralised service for maintaining configuration information in a distributed application.

The way zookeeper store its key-value pair is bit different than other key-value stores, Zookeeper uses z-node as a key. It looks like a unix filesystem tree and it starts with slash(/) It may be persistent or ephemeral. This key-value is served through RAM. Each node has its own ACL. Zookeeper stores transaction log and snapshot for recovering node in case of disaster, It is designed to behave as a fault tolerant and distributed k-v store, So it should be deployed as a cluster. A group of zookeeper server is called zookeeper ensemble. Here there is one zookeeper leader server and the remaining ones are the followers. This leader and follower relationship is derived from the leadership election between zk servers in a cluster.

Zookeeper is mainly used in HA implementation of Hadoop Namenode and YARN Resource manager here it takes care of promoting the active and standby status of these daemons, Kafka is designed to use Zookeeper for storing the topic and offset information.

Zookeeper can also be used as an alternative for etcd in kubernetes control plane.

Answer 2

You're comparing the high level data model of ZooKeeper to other key value stores, but that's not what makes it unique. From a distributed systems standpoint, ZooKeeper is different than many other key value stores (especially Redis) because it is strongly consistent and can tolerate failures while a majority of the cluster is connected. Additionally, while data is held in memory, it's synchroniusly replicated to a majority of the cluster and backed by disk, so once a write succeeds it guarantees that write will not be lost (barring a missile strike). This makes ZooKeeper very useful for storing small amounts of mission critical state like configurations.

Conversely, Redis is not a distributed system and does not provide the same sorts of guarantees that ZooKeeper does, and many other key value stores that are distributed are eventually consistent. In other words, there's no guarantee that once a value is written all other processes in a distributed system can see that value.

Finally, in addition to the file system like interface for storing state, ZooKeeper provides fairly low level features on which more complex problems can be solved. For examples of this look at Apache Curator. Curator uses ZooKeeper's ephemeral nodes (nodes that disappear when the client that created them disconnects) to build things like locks and leader elections which are extremely useful for coordinating distributed systems. So, from that perspective ZooKeeper's data model and associated features serve as primitives on which higher level tools for distributed coordination can be built.