The group coordinator is not available-Kafka

Question

When I am write a topic to kafka,there is an error:Offset commit failed:

2016-10-29 14:52:56.387 INFO [nioEventLoopGroup-3-1][org.apache.kafka.commo         


        

Answer 1

Looking at your logs the problem is that cluster probably don't have connection to node which is the only one know replica of given topic in zookeeper.

You can check it using given command:
kafka-topics.sh --describe --zookeeper localhost:2181 --topic test1

or using kafkacat:
kafkacat -L -b localhost:9092

Example result:

Metadata for all topics (from broker 1003: localhost:9092/1003):
 1 brokers:
  broker 1003 at localhost:9092
 1 topics:
  topic "topic1" with 1 partitions:
    partition 0, leader -1, replicas: 1001, isrs: , Broker: Leader not available

If you have single node cluster then broker id(1001) should match leader of topic1 partition.
But as you can see the only one known replica of topic1 was 1001 - which is not available now, so there is no possibility to recreate topic on different node.

The source of the problem can be an automatic generation of broker id(if you don't have specified broker.id or it is set to -1).
Then on starting the broker(the same single broker) you probably receive broker id different that previously and different than was marked in zookeeper (this a reason why partition deletion can help - but it is not a production solution).

The solution may be setting broker.id value in node config to fixed value - according to documentation it should be done on produciton environment:
broker.id=1

If everything is alright you should receive sth like this:

Metadata for all topics (from broker 1: localhost:9092/1001):
 1 brokers:
  broker 1 at localhost:9092
 1 topics:
  topic "topic1" with 1 partitions:
    partition 0, leader 1, replicas: 1, isrs: 1

Kafka Documentation: https://kafka.apache.org/documentation/#prodconfig