Kafka consumer fetching metadata for topics failed

Question

I am attempting to write a Java client for a third party\'s Kafka and ZooKeeper servers. I am able to list and describe topics, but when I attempt to read any, a Close

Answer 1

The broker tells the client which hostname should be used to produce/consume messages. By default Kafka uses the hostname of the system it runs on. If this hostname can not be resolved by the client side you get this exception.

You can try setting advertised.host.name in the Kafka configuration to an hostname/address which the clients should use.

Answer 2

Ran into this error on AWS. Problem was I was overly restrictive with the security group and set ports 2181 and 9092 to "my IP". This meant the kafka instance couldn't find the ZK running on the same box.

Solution - open it up - a little.

Answer 3

Here is my way to solve this problem:

1. run bin/kafka-server-stop.sh to stop running kafka server.
2. modify the properties file config/server.properties by adding a line: listeners=PLAINTEXT://{ip.of.your.kafka.server}:9092
3. restart kafka server.

Since without the lisener setting, kafka will use java.net.InetAddress.getCanonicalHostName() to get the address which the socket server listens on.

Answer 4

You have a problem with Zookeeper. 255.255.255.255:2181 is not a valid Zookeeper address; this is a broadcast address on your network or a subnet mask. To make the things work, find the IP Address or hostname of the machine running Zookeeper.