Zookeeper ensemble not coming up

Question

I am trying to configure the ensemble of 3 nodes following the documentation. All of them are on Linux Ubuntu. on all the three nodes configuration file looks like this :

Answer 1

Ensure that:

• you have started the Zookeeper server on all 3 servers
• all servers run in non-error mode by running echo ruok | netcat ip.of.zk2 2181. If ok, the server should respond with imok (FYI, here's a list of all supported 4-letter commands)
• /home/zkuser/zookeeper_data/myid contains values 1/2/3 for each server, respectively
• you can ping other 2 servers from the first server

If interested, I have created a vagrant+ansible script to create a 3-node virtual Zookeeper cluster, see https://github.com/mkrcah/virtual-zookeeper-cluster

Answer 2

I had a similar issue. I have got some hints of what the problem may be from here and here. In my case, the output of the command netstat -plutn was showing something including 127.0.0.1:3888 for the election port 3888. I have solved the problem by changing part of zoo.cfg on server n, from something like

server.1=name.of.s1:2888.3888
...
server.n=localhost:2888:3888
...

to

server.1=name.of.s1:2888.3888
...
server.n=0.0.0.0:2888:3888
...

After a ZooKeeper restart, the output of netstat -plutn includes :::3888.

Apparently this is needed for ZooKeeper to properly expose the election port, in this case 3888.