问题
I cannot connect to HBase running in Docker on Windows (banno/hbase-standalone image). However, I can connect to locally installed HBase.
banno/hbase-standalone image is run using:
docker run -d -p 2181:2181 -p 60000:60000 -p 60010:60010 -p 60020:60020 -p 60030:60030 banno/hbase-standalone
I also set up the port forwarding on the boot2docker-vm (which is required when running on Windows):
I can successfully telnet to all those ports on my localhost.
Next, here is a code sample that we use in our tests:
Configuration config = HBaseConfiguration.create();
config.clear();
config.setInt("timeout", 12000);
config.set("zookeeper.znode.parent", "/hbase");
config.set("hbase.zookeeper.quorum", "127.0.0.1");
config.set("hbase.zookeeper.property.clientPort", "2181");
config.set("hbase.master", "127.0.0.1:60000");
final Configuration configuration = HBaseConfiguration.create(config);
JobDefinition.Buildable.dumpProperties(configuration, newArrayList("hbase.*"));
HBaseAdmin.checkHBaseAvailable(config);
Which causes the following exception
Exception in thread "main" org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.net.UnknownHostException: unknown host: a3e6c240af20
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1651)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1677)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1885)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isMasterRunning(HConnectionManager.java:900)
at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:2366)
at com.xxx.compute.hadoop.jobs.transaction.OurTest.main(OurTest.java:24)
Caused by: com.google.protobuf.ServiceException: java.net.UnknownHostException: unknown host: a3e6c240af20
at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1674)
at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1715)
at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(HConnectionManager.java:1688)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1597)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1623)
... 5 more
Caused by: java.net.UnknownHostException: unknown host: a3e6c240af20
at org.apache.hadoop.hbase.ipc.RpcClient$Connection.<init>(RpcClient.java:386)
at org.apache.hadoop.hbase.ipc.RpcClient.createConnection(RpcClient.java:352)
at org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1526)
at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1438)
at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1657)
... 10 more
This is explainable. We run Windows, which requires boot2docker-vm virtual machine running using NAT. The Docker container of the image is running inside the boot2docker-vm also using NAT. However, the ports are "visible" to the host machine running tests, since Docker container exports the ports, and the boot2docker-vm forwards the ports the host machine. The name a3e6c240af20 actually comes from the Docker container ID, so probably a3e6c240af20 is a hostname for the Docker container :
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a3e6c240af20 banno/hbase-standalone:latest "/bin/sh -c '/opt/hb 24 minutes ago Up 24 minutes 0.0.0.0:2181->2181/tcp, 0.0.0.0:60000->60000/tcp, 0.0.0.0:60010->60010/tcp, 0.0.0.0:60020->60020/tcp, 0.0.0.0:60030->60030/tcp agitated_wozniak
I am not sure how exactly HBase communication works, but apparently it makes RPC calls to the instance. HBase Docker returns its hostname hoping that the client will call it there. But since both boot2docker-vm and Docker container running using NAT, the host machine does not see the Docker container.
I tried to add a3e6c240af20 to my hosts file:
127.0.0.1 a3e6c240af20
Then I get a different error, also during the RPC call, which actually does not help me much:
Exception in thread "main" org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.lang.NullPointerException
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1651)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1677)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1885)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isMasterRunning(HConnectionManager.java:900)
at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:2366)
at com.xxx.compute.hadoop.jobs.transaction.OurTest.main(OurTest.java:24)
Caused by: com.google.protobuf.ServiceException: java.lang.NullPointerException
at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1674)
at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1715)
at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(HConnectionManager.java:1688)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1597)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1623)
... 5 more
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hbase.ipc.RpcClient$Connection.writeRequest(RpcClient.java:1051)
at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1440)
at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1657)
... 10 more
Does anyone have a suggestion how this can be solved?
回答1:
Try add [boot2docker IP] a3e6c240af20 instead of 127.0.0.1 because HBase Java client needs to reach your docker's host not exactly localhost to reach zookeeper (CMIIW). Not pretty sure if it will works but it works in my Windows.
回答2:
I used oddpoet/hbase-cdh5 docker image to avoid this issue.
docker run -d -p 2181:2181 -p 60000:60000 -p 60010:60010 -p 60020:60020 -p 60030:60030 -h hbase oddpoet/hbase-cdh5
fig.yml
hbase:
image: oddpoet/hbase-cdh5
hostname: hbase
ports:
- "3181:2181"
- "60000:60000"
- "60010:60010"
- "60020:60020"
- "60030:60030"
my configuration file
conf.set("hbase.zookeeper.quorum", zkPath);
conf.set("hbase.zookeeper.property.clientPort","2181");
conf.set("zookeeper.znode.parent", "/hbase");
conf.set("hbase.client.retries.number", "3"); // default 35
conf.set("hbase.rpc.timeout", "10000"); // default 60 secs
conf.set("hbase.rpc.shortoperation.timeout", "5000"); // default 10 secs
来源：https://stackoverflow.com/questions/25564097/connect-to-hbase-running-in-docker