How to reset the retry count in Spring Kafka consumer when the exception thrown in the first retry is different from the second retry?

问题

I am trying to implement a Kafka retry consumer in spring-boot and using SeekToCurrentErrorHandler for the retries.

I have set the backoff policy to have 5 retry attempts.

My question is, lets say the first attempt of the retry, the exception was 'database not available', and the second attempt db was available but there is another failure at another step like a timeout, in this case will the retry count goes back to zero and starts a fresh or will continue to try only of the remaining attmepts from the 1st retry.

My requirement is to reset the retry count back to zero everytime an exception is different to what was thrown previously.

How to achieve that in Kafka?

Here is my consumer config.

@Bean
    public ConcurrentKafkaListenerContainerFactory<String, Anky> kafkaRetryListenerContainerFactory() {

        ConcurrentKafkaListenerContainerFactory<String, Anky> factory = new ConcurrentKafkaListenerContainerFactory<>();
        factory.setConsumerFactory(consumerFactory());
        factory.getContainerProperties().setAckMode(AckMode.MANUAL_IMMEDIATE);

        SeekToCurrentErrorHandler errorHandler = new SeekToCurrentErrorHandler((record, exception) -> {
            System.out.println(
                    "RetryPolicy** limit has been exceeded! You should really handle this better." + record.key());
        }, new FixedBackOff(0L, 5L));
        errorHandler.addNotRetryableException(IllegalArgumentException.class);
        errorHandler.setCommitRecovered(true);

        factory.setErrorHandler(errorHandler);
        // to keep the consumers alive the failure gets reported to broker so that
        // consumers remain alive
        factory.setStatefulRetry(true);
        factory.setConcurrency(2);
        return factory;
    }

@KafkaListener(topics = "${retry.topic}", groupId = "${consumer.groupId}", containerFactory = "kafkaRetryListenerContainerFactory")
    public void onRetryMessage(ConsumerRecord<String, Anky> record, Acknowledgment acknowledgment)
            throws SystemException {
        LOGGER.debug("are you here in retry?***");
//process message has both DB calls as well as rest calls so it can fail due to a database error and can also fail due to HTTP failure
            processMessage(record, acknowledgment, false, offerIntakeContext);
            }
            
    }

回答1:

That's not possible with the standard error handler; it doesn't keep track of the previous exception type.

You would need a custom error handler; the only other option is to keep track of the previous exception in your listener, and call clearThreadState() on the error hander (from the listener method, on the listener thread) which will clear all retry state for this listener thread.

来源：https://stackoverflow.com/questions/64708287/how-to-reset-the-retry-count-in-spring-kafka-consumer-when-the-exception-thrown

标签

spring-boot

apache-kafka

kafka-consumer-api

spring-kafka