Kafka behind Traefik on Kubernetes

问题

I am trying to configure a Kafka cluster behind Traefik but my producers and client (that are outside kubernetes) don't connect to the bootstrap-servers. They keep saying:

"no resolvable boostrap servers in the given url"

Actually here is the Traefik ingress:

{
    "apiVersion": "extensions/v1beta1",
    "kind": "Ingress",
    "metadata": {
        "name": "nppl-ingress",
        "annotations": {
            "kubernetes.io/ingress.class": "traefik",
            "traefik.frontend.rule.type": "PathPrefixStrip"
        }
    },
    "spec": {
        "rules": [
            {
                "host": "" ,
                "http": {
                    "paths": [
                        {
                            "path": "/zuul-gateway",
                            "backend": {
                                "serviceName": "zuul-gateway",
                                "servicePort": "zuul-port"
                            }
                        },                      
                        {
                            "path": "/kafka",
                            "backend": {
                                "serviceName": "kafka-broker",
                                "servicePort": "kafka-port"
                            }

[..]
    }

What I give to the kafka consumers/producers is the public IP of Traefik. Here is the flow: [Kafka producers/consumers] -> Traefik(exposed as Load Balancer) -> [Kafka-Cluster]

Is there any solution? Otherwise I was thinking to add a kafka-rest proxy (https://docs.confluent.io/current/kafka-rest/docs/index.html) between Traefik and the kafka brokers but I think isn't the ideal solution.

回答1:

I did. You can refer to it, in kubernetes ,deployment kafka.yaml

   env:
    - name: KAFKA_BROKER_ID
      value: "1"
    - name: KAFKA_CREATE_TOPICS
      value: "test:1:1"
    - name: KAFKA_ZOOKEEPER_CONNECT
      value: "zookeeper:2181"
    - name: KAFKA_ADVERTISED_LISTENERS
      value: "INSIDE://:9092,OUTSIDE://kafka-com:30322"
    - name: KAFKA_LISTENERS
      value: "INSIDE://:9092,OUTSIDE://:30322"
    - name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
      value: "INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT"
    - name: KAFKA_INTER_BROKER_LISTENER_NAME
      value: "INSIDE"

kafka service,the external service invocation address, or traefik proxy address

---
kind: Service
apiVersion: v1
metadata:
  name: kafka-com
  namespace: dev
  labels:
    k8s-app: kafka
spec:
  selector:
    k8s-app: kafka
  ports:
  - port: 9092
    name: innerport
    targetPort: 9092
    protocol: TCP
  - port: 30322
    name: outport 
    targetPort: 30322
    protocol: TCP
    nodePort: 30322
  type: NodePort

Ensure that Kafka external port and nodePort port are consistent,Other services call kafka-com:30322, my blog write this config_kafka_in_kubernetes, hope to help U !

来源：https://stackoverflow.com/questions/52798091/kafka-behind-traefik-on-kubernetes

标签

Kubernetes

apache-kafka

load-balancing

Traefik