Kafka consumer fails to consume if first broker is down

Question

I\'m using latest version of kafka(kafka_2.12-1.0.0.tgz). I have setup simple cluster with 3 brokers(just changed broker.id=1 and listeners=PLAINTEXT://:9092 in properties f

Answer 1

Try to check "offsets.topic.replication.factor" in server-*.properties file

For example:

############################# Internal Topic Settings       
# The replication factor for the group metadata internal topics    
# For anything other than development testing, a value greater than 1 is  recommended for to ensure availability such as 3.
offsets.topic.replication.factor=3

http://kafka.apache.org/documentation/#brokerconfigs

Answer 2

Using KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR on yml file solve this issue.

E.g. Using 2 workers on docker-swarm.

environment:
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 2