Consumer not receiving messages, kafka console, new consumer api, Kafka 0.9

Question

I am doing the Kafka Quickstart for Kafka 0.9.0.0.

I have zookeeper listening at localhost:2181 because I ran

bin/zookeeper-server-start.sh          


        

Answer 1

In my case it didn't worked using either approaches then I also increased the log level to DEBUG at config/log4j.properties, started the console consumer

./bin/kafka-console-consumer.sh --bootstrap-server 127.0.0.1:9092 --from-beginning --topic MY_TOPIC

Then got the log below

[2018-03-11 12:11:25,711] DEBUG [MetadataCache brokerId=10] Error while fetching metadata for MY_TOPIC-3: leader not available (kafka.server.MetadataCache)

The point here is that I have two kafka nodes but one is down, by some reason by default kafka-console consumer will not consume if there is some partition not available because the node is down (the partition 3 in that case). It doesn't happen in my application.

Possible solutions are

• Startup the down brokers
• Delete the topic and create it again that way all partitions will be placed at the online broker node