If I have Transactional Producer in Kafka can I read exactly once messages with Kafka Streams?

走远了吗. 提交于 2020-03-26 04:28:29

问题


I would like to have Exactly-once semantics, but I don't want to read message with Consumer. I'd rather read messages with Kafka Streams AP. If I add processing.guarantee=exactly_once to Stream configuration, will exactly-once semantics be kept?


回答1:


Exactly-once processing is based on a read-process-write pattern. Kafka Streams uses this pattern and thus, if you write a regular Kafka Streams application that writes the result back to a Kafka topic, you will get exactly-once processing guarantees.

Note, that side-effects are not covered by this guarantee. Also note, that there might be retries internally in case of failure. Exactly-once implies, that the result you see in the output topic, is the same as if no error (and thus not retries) would have happened.

For more details, you might want to watch a talk recording about exactly-once guarantees in Kafka. There are multiple available at Confluent webpage (disclaimer: I am an employee at Confluent): https://www.confluent.io/resources/



来源:https://stackoverflow.com/questions/50698695/if-i-have-transactional-producer-in-kafka-can-i-read-exactly-once-messages-with

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!