Spring Kafka SeekToCurrentErrorHandler Find Out Which Record Has Failed

Question

I have implemented a Kafka consumer with KafkaHandler. My consumer is supposed to consume events, then send a REST request to some other service for each event. I w

Answer 1

There is a FailedRecordTracker since Spring for Apache Kafka 2.2 (not released yet):

https://docs.spring.io/spring-kafka/docs/2.2.0.M2/reference/html/whats-new-part.html#_listener_container_changes

Starting with version 2.2, the SeekToCurrentErrorHandler can now recover (skip) a record that keeps failing. By default, after 10 failures, the failed record will be logged (ERROR). You can configure the handler with a custom recoverer (BiConsumer) and/or max failures.

SeekToCurrentErrorHandler errorHandler =
    new SeekToCurrentErrorHandler((record, exception) -> {
          // recover after 3 failures - e.g. send to a dead-letter topic
          }, 3);

So, what you need is just copy/paste a FailedRecordTracker and SeekToCurrentErrorHandler source code from the master into your project and you will have a functionality you are seeking for:

https://github.com/spring-projects/spring-kafka/blob/master/spring-kafka/src/main/java/org/springframework/kafka/listener/FailedRecordTracker.java

https://github.com/spring-projects/spring-kafka/blob/master/spring-kafka/src/main/java/org/springframework/kafka/listener/SeekToCurrentErrorHandler.java