I can't produce data to Kafka when use the script, But I can list topics with the script

问题

everybody,there is a virtual server in the local area network which ip is 192.168.18.230, and my machine ip is 192.168.0.175. Today, I try to use my machine (192.168.0.175) to send some messages to my virtual server(192.168.18.230), with the Kafka console producer

$ bin/kafka-console-producer.sh --broker-list 192.168.18.230:9092 --topic test

but there is something wrong. The description of the problem is :

[2017-04-10 17:25:40,396] ERROR Error when sending message to topic test with key: null, value: 6 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for test-0 due to 1568 ms has passed since batch creation plus linger time

But when I use the kafka-topics script to list topics, it works:

$ bin/kafka-topics.sh --list --zookeeper 192.168.18.230:2181

This problem confused me a very long period, can any body help me to solve it?

回答1:

If you have a zookeeper instance which is running, you can of course ask the list of topics. However, it seems that you have no Kafka broker available.

You maybe have a zookeeper running but not Kafka.

回答2:

Your Kafka producer might be running on a machine which cannot access your virtual machine where your Kafka broker is running.

Also, not only should the broker port be open, but also it must be answerable by the broker i.e, the (advertised) listeners of your Kafka broker must have your virtual machine IP (IP accessible from where your Kafka producer is running, because a VM can have multiple IPs and there is no rule that all IPs will be accessible).

For example, your virtual machine have two IPs 1.2.3.4 and 4.3.2.1 and your producer on another machine points to 1.2.3.4, you must be able first to ping and telnet to this IP.

Next, you must have this IP 1.2.3.4 in the advertised listeners in your Kafka broker.

advertised.listeners=PLAINTEXT://1.2.3.4:9092

You should set this IP only as your bootstrap.servers in your Kafka Producer.

You should also ensure that the port is not just open to localhost or 127.0.0.1 like for example, when you do netstat, it should not have just localhost:9092 or 127.0.0.1:9092, it should use any local address 0.0.0.0 or your IP 1.2.3.4:9092

来源：https://stackoverflow.com/questions/43319549/i-cant-produce-data-to-kafka-when-use-the-script-but-i-can-list-topics-with-th