Transaction Synchronization in Spring Kafka

Question

I want to synchronize a kafka transaction with a repository transaction:

@Transactional
public void syncTransaction(){
  myRepository.save(someObject)
  kafka         


        

Answer 1

@Eike Behrends to have a db + kafka transaction, you can use ChainedTransactionManager and define it this way :

@Bean
public KafkaTransactionManager kafkaTransactionManager() {
    KafkaTransactionManager ktm = new KafkaTransactionManager(producerFactory());;
    ktm.setTransactionSynchronization(AbstractPlatformTransactionManager.SYNCHRONIZATION_ON_ACTUAL_TRANSACTION);
    return ktm;
}


@Bean
@Primary
public JpaTransactionManager transactionManager(EntityManagerFactory em) {
    return new JpaTransactionManager(em);
}

@Bean(name = "chainedTransactionManager")
public ChainedTransactionManager chainedTransactionManager(JpaTransactionManager jpaTransactionManager,
                                                           KafkaTransactionManager kafkaTransactionManager) {
    return new ChainedTransactionManager(kafkaTransactionManager, jpaTransactionManager);
}

You need to annotate your transactional db+kafka methods @Transactional("chainedTransactionManager")

(you can see the issue on spring-kafka project : https://github.com/spring-projects/spring-kafka/issues/433 )

You say :

From my perspective it looks weird that I send an event via kafkaTemplate, listen to my own event and forward the event using the kafkaTemplate again.

Have you tried this ? If so can you provide an example please ?

Answer 2

If the listener container is provisioned with a KafkaTransactionManager, the container will create a producer which will be used by any downstream kafka template and the container will send the offsets to the transaction for you.

If the container has some other transaction manager, the container can't send the offsets since it doesn't have access to the producer (or template).

Another solution is to annotate your method with @Transactional (with the datasource TM) and configure the container with a kafka TM.

That way, your DB tx will commit just before the thread returns to the container which will then send the offsets to the kafka transaction and commit it.

See the framework test cases for examples.