confluent-kafka

Sometimes a new consumer group does not work

阅读更多关于 Sometimes a new consumer group does not work

问题 I've seen this in production once (I don't remember how we solved it) and now I can repeat it in the integration tests, which always start with a brand new Kafka installation. Here's how it goes: Step 1: A consumer of a group that doesn't exist yet subscribes to a topic that does not exist yet and starts polling. self.kafka_consumer = confluent_kafka.Consumer({ 'group.id': 'mygroup', 'bootstrap.servers': 'kafka:9092', 'enable.auto.commit': False, 'auto.offset.reset': 'earliest', }) self.kafka

Kafka Configuration for only seeing last 5 minutes of data

阅读更多关于 Kafka Configuration for only seeing last 5 minutes of data

Sorry i am new in Kafka and this question migth be so easy but i need some help. i did not figure out some configurations. There is a stream data, i want Consumers to see only last 5 minutes of messages that procuders sent. I am using Confluent.Kafka for .Net, var config = new Dictionary<string, object>{ {"group.id","Test1Costumers"}, {"bootstrap.servers",brokerEndpoint}, { "auto.commit.interval.ms", 60000}, { "auto.offset.reset", "earliest" } }; Here is config dictionary of Consumers in github example, another issue is i dont want to store messages in a topic more than 5 minutes cos i wont

订阅 confluent-kafka