Hbase client ConnectionLoss for /hbase error

Question

I\'m going completely crazy:

Installed Hadoop/Hbase, all is running;

/opt/jdk1.6.0_24/bin/jps
23261 ThriftServer
22582 QuorumPeerMain
21969 NameNode
23         


        

Answer 1

This is a Zookeeper(ZK) error. The HBase client tries to get the /hbase node from Zookeeper and fails.

You can get a ZK dump from the HBase master web interface. You should see all the connections to ZK and figure out if something is exhausting them.

Before diving into anything else you could try restarting your ZK cluster and see if it fixes your problem. (It's strange that you see that with a single client).

HBase has a setting to increase the number of connections to ZK. It's

hbase.zookeeper.property.maxClientCnxns

There were a few updates (see below) lately related to the default number of connections (there's a hbase-default.xml file that has all the default configurations). You can override this in your hbase-site.xml file (under HBase conf dir) and raise it to 100 or more. But make sure you're not masking the real problem this way, you shouldn't see this problem with a single client.

We've had a similar situation, but it was happening during heavy operations from map-reduce jobs, after upgrading to HBase-0.90.

Here are a couple of issue related to your problem:

• https://issues.apache.org/jira/browse/HBASE-3773
• https://issues.apache.org/jira/browse/HBASE-3777

If you still can't figure it out send an email to the hbase-users list or join the #hbase channel on freenode and ask live questions.