问题
I am currently struggling with the following problem:
I have a Java EE application that shall be able to run in a cluster (it actually does). For the data exchange between the nodes I use Hazelcast 3.3.3. Since our customers are afraid of UDP we use TCP. So I configure 3 hazelcast parameters per node: The network interface to use, the port and the members of the cluster. In real life (not a docker environment) everything works. Now I want to put my Java EE servers into docker containers (one per node) and also link them via Hazelcast.
What I did was:
On the host machine running the docker containers I created several virtual network interfaces (172.21.0.X). for each docker container I gave network mappings like -p 172.21.0.X:5190:5190 (5190 is the port I use for Hazelcast)
In my Java EE application within the container I configured hazelcast to listen to interface 172.21.0.X, port 5190 members 172.21.0.X:5190,...... The problem now is: docker will assign an internal IP number to each cointainer. In advance I cannot be sure which IP number docker will assign. It definitly is not the IP I would like to use (172.21.0.X). Hazelcast starts up and tells me: no matching interface found. Of course not. Hazelcast will check the IP interfaces of the docker container which is something like 172.4.XX. This does not fit to the config I gave to hazelcast (use 172.21.X.X).
So far it seems not to be possible to specify the internal IP number of a docker container. I could query it from a running container, but that is too late since I need to do the configuration of my Java EE server BEFORE I start the container.
Any hints on that? Is it possible to get the Hazelcast running with TCPIP?
回答1:
This thread and the answer from Nick Scavelli is what I think you are looking for.
https://groups.google.com/forum/#!msg/vertx/MvKcz_aTaWM/QM6CfllX9HQJ
回答2:
This thread provides an answer and an example: https://groups.google.com/d/msg/orient-database/ehfKcInWZXs/wxs8xLUOGB8J
Basically you use hazelcast variables in your hazelcast file, i.e.
<property name="hazelcast.local.localAddress">${hazelcast.ip}</property>
Then when you're running the docker contained you add:
-e IP=`ip route | awk '/default/ { print $9 }'`
Which sets an environment variable in the docker container. Then in there you do something like this:
CMD ["java", "-Dhazelcast.ip=${IP}", ...
So it's taking the docker env variable, and smashing into java as a definition, which hazelcast then picks out and substitutes into it's config file at runtime.
回答3:
I want to make @gremwell's answer up-to-date, because now it's not quite correct. Property hazelcast.local.localAddress
is no longer valid. Instead one should put this snippet inside hazelcast.xml
<network>
<public-address>${hazelcast.ip}:${hazelcast.port}</public-address>
<port auto-increment="false" port-count="100">${hazelcast.port}</port>
...
</network>
Then pass the params while starting docker container (I use official hazelcast image and mount folder with my config using -v flag)
sudo docker run -p 5701:5701 -e JAVA_OPTS="-Dhazelcast.config=/etc/conf/hazelcast.xml -Dhazelcast.ip=`ip route get 8.8.8.8 | awk '{print $NF; exit}'` -Dhazelcast.port=5701" -v /data/docker/hazelcast:/etc/conf -ti hazelcast/hazelcast
Command -Dhazelcast.ip=ip route get 8.8.8.8 | awk '{print $NF; exit}'
takes host machine ip and pass it inside config file as hazelcast.ip variable.
Inner port and docker port mapping should be similar 5701 in 3 places. Next container run with 5702 and so on.
Hope this helps.
来源:https://stackoverflow.com/questions/30103306/hazelcast-in-multinode-docker-environments-with-tcpip