Kafka configuration min.insync.replicas not working

∥☆過路亽.° 提交于 2020-01-06 08:11:40

问题


Its my early days in learning kafka. And I am checking out every kafka property/concept in my local machine.

So I came across this property min.insync.replicas and here is my understanding. Please correct me if I've misunderstood anything.

  • Once a message is sent to a topic, the message must be written to at least min.insync.replicas number of followers.
  • min.insync.replicas also includes the leader.
  • If number of available live brokers( indirectly, in sync replicas ) are less than the specified min.insync.replicas , then producer will raise an exception failing to publish the message.

Following are the steps I followed to create the above scenario

  1. Started 3 brokers in local with broker Ids 0, 1 and 2
  2. created the topic insync and set min.insync.replicas to 2 using the following command

sudo ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic insync --config min.insync.replicas=2

  1. Describe the topic resulted in the following

Topic:insync PartitionCount:1 ReplicationFactor:3 Configs:min.insync.replicas=2 Topic: insync Partition: 0 Leader: 2 Replicas: 2,0,1 Isr: 1,2,0

At this point, I made sure the property I've provided is picked by kafka

  1. I started sending messages and consuming them from terminal using following command

    Producer: ./kafka-console-producer.sh --broker-list localhost:9092 --topic insync --producer.config ../config/producer.properties

    Consumer: ./kafka-console-consumer.sh --zookeeper localhost:2181 --topic insync

At this point, I was able to send and receive messages successfully.

  1. Bought down 2 brokers (0 and 2) and described the topic and resulted in following

Topic:insync PartitionCount:1 ReplicationFactor:3 Configs:min.insync.replicas=2 Topic: insync Partition: 0 Leader: 1 Replicas: 2,0,1 Isr: 1

At this point, the In Sync Replicas are just 1(Isr: 1)

Then I tried to produce the message and it worked. I was able to send messages from console-producer and I could see those messages in console consumer.

My Kafka version: kafka_2.10-0.10.0.0

following are the producer properties:

bootstrap.servers=localhost:9092

compression.type=none

batch.size=20

acks=all

I expected the producer to fail with NotEnoughReplicasException as mentioned in this.

public class NotEnoughReplicasException extends RetriableException

Number of insync replicas for the partition is lower than >min.insync.replicas

but it worked normally.

Am I missing something? How can I create the scenario?

*************** EDIT **********************

Instead of producing the messages from console producer, I tried to generate messages from java code. This time, I got the expected exception in the kafka broker. Although I expected it in the producer (java code). As this experiment is raising more questions, I've posted another question.


回答1:


is acks set to "all"? if not, try setting it to all



来源:https://stackoverflow.com/questions/58780219/kafka-configuration-min-insync-replicas-not-working

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!