Consumer not receiving messages, kafka console, new consumer api, Kafka 0.9

Question

I am doing the Kafka Quickstart for Kafka 0.9.0.0.

I have zookeeper listening at localhost:2181 because I ran

bin/zookeeper-server-start.sh          


        

Answer 1

Can you please try like this:

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --from-beginning --topic my-replicated-topic

Answer 2

I just ran into this issue and the solution was to delete /brokers in zookeeper and restart the kafka nodes.

bin/zookeeper-shell <zk-host>:2181

and then

rmr /brokers

Not sure why this solves it.

When I enabled debug logging, I saw this error message over and over again in the consumer:

2017-07-07 01:20:12 DEBUG AbstractCoordinator:548 - Sending GroupCoordinator request for group test to broker xx.xx.xx.xx:9092 (id: 1007 rack: null) 2017-07-07 01:20:12 DEBUG AbstractCoordinator:559 - Received GroupCoordinator response ClientResponse(receivedTimeMs=1499390412231, latencyMs=84, disconnected=false, requestHeader={api_key=10,api_version=0,correlation_id=13,client_id=consumer-1}, responseBody={error_code=15,coordinator={node_id=-1,host=,port=-1}}) for group test 2017-07-07 01:20:12 DEBUG AbstractCoordinator:581 - Group coordinator lookup for group test failed: The group coordinator is not available. 2017-07-07 01:20:12 DEBUG AbstractCoordinator:215 - Coordinator discovery failed for group test, refreshing metadata

Answer 3

In kafka_2.11-0.11.0.0 the zookeeper server is deprecated and and it is using bootstrap-server, and it will take broker ip address and port. If you give correct broker parameters you will be able to consume messages.

e.g. $ bin/kafka-console-consumer.sh --bootstrap-server :9093 --topic test --from-beginning

I'm using port 9093, for you it may vary.

regards.

Answer 4

Run the below command from bin:

./kafka-console-consumer.sh --topic test --from-beginning --bootstrap-server localhost:9092

"test" is the topic name

Answer 5

I my MAC box I was facing the same issue of console-consumer not consuming any messages when used the command

kafka-console-consumer --bootstrap-server localhost:9095 --from-beginning --topic my-replicated-topic

But when I tried with

kafka-console-consumer --bootstrap-server localhost:9095 --from-beginning --topic my-replicated-topic --partition 0

It happily lists the messages sent. Is this a bug in Kafka 1.10.11?

Answer 6

Was getting the same issue on my Mac. I checked the logs and found the following error.

Number of alive brokers '1' does not meet the required replication factor '3' for the offsets topic (configured via 'offsets.topic.replication.factor'). 
This error can be ignored if the cluster is starting up and not all brokers are up yet.

This can be fixed by changing the replication factor to 1. Add the following line in server.properties and restart Kafka/Zookeeper.

offsets.topic.replication.factor=1