Apache Kafka example error: Failed to send message after 3 tries

Question

I am running this kafka producer example mentioned in its site

The code:

public class TestProducer {

    public static void main(String[] args) {
               


        

Answer 1

This is a solution to the exception in the original question asked by Krish: "kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries."

The FAQ here and here says that your hostname should be set correctly. I have not experienced that condition. But I found another condition when a kafka producer gives this error message: when your partition key in the producer is wrong. That is, if you have a topic with one partition, then partition key in the producer can be either null(message is sent to a random partition) or 0(partitions in kafka are numbered starting from 0). If you try to use a partition key of 1, this exception is thrown in the producer. Or if you have 3 partitions in the topic, and you use a partition key of 3(key of 3 is invalid because valid partition numbers are 0,1,2), this exception is thrown. This error is consistent when the partition number in the producer's send() method does not match with the range of partitions in the topic. I used kafka version 0.8.2. The client API I used was the package kafka.javaapi.producer.Producer.

Answer 2

ForHDP kafka use broker port: 6667

For Standalone kafka use broker port: 9092

The error was because of the port no which we were using (HDP uses 6667 but we were using 9092)

bin/kafka-console-producer.sh --broker-list broker-ip:9092 --topic test //not working

bin/kafka-console-producer.sh --broker-list broker-ip:6667 --topic test //working

link: Kafka console producer Error in Hortonworks HDP 2.3 Sandbox