Kafka in Kubernetes Cluster- How to publish/consume messages from outside of Kubernetes Cluster

Question

1. I have Kafka deployed and running in Kubernetes cluster. I am using this image from docker hub - https://hub.docker.com/r/cloudtrackinc/kubernetes-kafka/
2. I have 3

Answer 1

This seems not to be possible at the moment, the network architecture of kafka is pretty poor regarding to this topic. The new consumer uses a list of brokers, which return the host of the zookeeper, but unfortunately this is in a different network, so it is not possible to reach it from your local client. The poor part of kafka is, that is not possible to specify the brokers AND the zookeeper servers. This prevents clients accessing the system from outside.

We worked around this for the moment using a busybox, where we installed tools to interact with kafka. In our case plunger