ERROR Error when sending message to topic

Question

While producing message in kafka, i am getting the following error :

$ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic nil_PF1_P1
hi
hello         


        

Answer 1

Another scenario here. No clue on what was happening till I find a kafka log with the following message:

Caused by: java.lang.IllegalArgumentException: Invalid version for API key 3: 2

Apparently the producer was using a newer kafka-client (java) than the kafka server and the API used was invalid (client using 1.1 and server on 10.0). On the Client/Producer I got the:

Error producing to topic Failed to update metadata after 60000 ms.

Answer 2

Had this issue: Using Hortonworks HDP 2.5. Kerberisation enabled

Fixed by providing the correct security protocol and ports. Example commands:

./kafka-console-producer.sh --broker-list sand01.intranet:6667, san02.intranet:6667, san03.intranet:6667--topic test--security-protocol PLAINTEXTSASL


./kafka-console-consumer.sh --zookeeper sand01:2181 --topic test--from-beginning --security-protocol PLAINTEXTSASL

Answer 3

Instead of changing server.properties include the address 0.0.0.0 in the code itself. Instead of

/usr/bin/kafka-console-producer --broker-list Hostname:9092 --topic MyFirstTopic1

use

/usr/bin/kafka-console-producer --broker-list 0.0.0.0:9092 --topic MyFirstTopic1

Answer 4

For Apache Kafka v2.11-1.1.0

Start zookeeper server:

$ bin/zookeeper-server-start.sh config/zookeeper.properties

Start kafka server:

$ bin/kafka-server-start.sh config/server.properties

Create one topic name "my_topic":

$ bin/kafka-topics.sh --create --topic my_topic --zookeeper localhost:2181 --replication-factor 1 --partitions 1

Start the producer:

$ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic my_topic

Start the consumer:

$ bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic my_topic --from-beginning

Answer 5

It may be because of some parameters from Kafka's server.properties file. You can find more information here

1. Stop the Kafka server with

   cd $KAFKA_HOME/bin  
   ./kafka-server-stop.sh
   

2. Change

   listeners=PLAINTEXT://hostname:9092   
   

   to

   listeners=PLAINTEXT://0.0.0.0:9092

   in $KAFKA_HOME/config/server.properties

3. Restart the Kafka server with

   $KAFKA_HOME/bin/kafka-server-start.sh $KAFKA_HOME/config/server.properties  
   

Answer 6

In my case, I am using Kafka docker with Openshift. I was getting the same problem. It got fixed when I passed the environment variable KAFKA_LISTENERS with a value of PLAINTEXT://:9092. This will eventually add create an entry listeners=PLAINTEXT://:9092 under server.properties.

The listeners doesn't have to have a hostname.