问题
I have Kafka Stream application. My application is processing the events successfully.
How to change Kafka committed consumer offset with required offset to reprocess/skip the events. I tried How to change start offset for topic?. But I got 'Node does not exist:' error. Please help me.
回答1:
The question/answer you are referring to is based on an older Kafka version. Since Kafka 0.9, offsets are not committed to ZooKeeper but store in a special Kafka topic called the offset topic (topic name is __consumer_offsets
).
Since Kafka 1.0, the command line tool bin/kafka-consumer-groups.sh
has a new feature that allow to set offsets. Check out the original KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-122%3A+Add+Reset+Consumer+Group+Offsets+tooling
The tool also works for Kafka 0.11 (and maybe even older Kafka versions).
An alternative is to write your own tool, that uses an single KafkaConsumer
with the corresponding group.id
, subscribe to the topic(s) you want to modify offsets for, seek()
and commit()
offsets. (Note, you should disable auto commit for this consumer.)
来源:https://stackoverflow.com/questions/47472532/how-change-kafka-committed-consumer-offset-with-required-offset