KAFKA - What is reason for getting ProducerFencedException during producer.send

Question

Trying to load around 50K messages into KAFKA topic. In the beginning of few runs getting below exception but not all the time.

org.apache.kafka.common.Kafka         


        

Answer 1

When running multiple instances of the application, transactional.id must be the same on all instances to satisfy fencing zombies when producing records on a listener container thread. However, when producing records using transactions that are not started by a listener container, the prefix has to be different on each instance.

https://docs.spring.io/spring-kafka/reference/html/#transaction-id-prefix