Kafka in Kubernetes Cluster- How to publish/consume messages from outside of Kubernetes Cluster

Question

1. I have Kafka deployed and running in Kubernetes cluster. I am using this image from docker hub - https://hub.docker.com/r/cloudtrackinc/kubernetes-kafka/
2. I have 3

Answer 1

I was able to solve my problem by doing the following changes -

1. Using NodeSelector in YML to make kafka pod run on a particular node of kube cluster.

2. Set KAFKA_ADVERTISED_HOST_NAME to Kube hostName where this Kafka POD has been configured to run on ( as configured in step 1 )

3. Expose Kafka Service using NodePort and set POD port same as that of exposed NodePort as shown below -

   spec:
     ports:
       - name: broker-2
         port: **30031**
         targetPort: 9092
         nodePort: **30031**
         protocol: TCP
     selector:
       app: kafka-2
       broker_id: "2"
     type: NodePort
   

Now, you can access Kafka brokers from outside of kube cluster using host:exposedPort