spring-kafka

问题 I am trying to see if the error channel I am passing to @ServiceActivator can be bounded/parameterized referring the value specified in YAML instead of hardcoding actual destination and consumer group in the code itself. @ServiceActivator( // I do not want to hardcode destination and consumer group here inputChannel = "stream-test-topic.my-consumer-group.errors" ) public void handleError(ErrorMessage errorMessage) { // Getting exception objects Throwable errorMessagePayload = errorMessage

问题 I am trying to see if the error channel I am passing to @ServiceActivator can be bounded/parameterized referring the value specified in YAML instead of hardcoding actual destination and consumer group in the code itself. @ServiceActivator( // I do not want to hardcode destination and consumer group here inputChannel = "stream-test-topic.my-consumer-group.errors" ) public void handleError(ErrorMessage errorMessage) { // Getting exception objects Throwable errorMessagePayload = errorMessage

问题 I am building a Kafka Consumer application that consumes messages from a Kafka Topic and performs a database update task. The messages are produced in a large batch once every day - so the Topic has about 1 million messages loaded in 10 minutes. The Topic has 8 partitions. The Spring Kafka Consumer (annotated with @KafkaListener and using a ConcurrentKafkaListenerContainerFactory) is triggered in very short batches. The batch size is sometimes just 1 or 2 messages. It would help performance

问题 Our kafka system crashed because no disk space was available. The consumers are Spring boot application which are using the Kafka Streams API. Now every consumer application shows the following error: java.io.FileNotFoundException: /tmp/kafka-streams/908a79bc-92e7-4f9c-a63a-5030cf4d3555/streams.device-identification-parser/0_48/.checkpoint.tmp (No such file or directory) This exception occurred exactly after the kafka server was restarted. If we restart the application, the service starts at

问题 I want to implement Kafka Consumer and Producer which sends and receives Java Objects. Full Source I tried this: Producer: @Configuration public class KafkaProducerConfig { @Value(value = "${kafka.bootstrapAddress}") private String bootstrapAddress; @Bean public ProducerFactory<String, SaleRequestFactory> saleRequestFactoryProducerFactory() { Map<String, Object> configProps = new HashMap<>(); configProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapAddress); configProps.put

问题 I want to implement Kafka Consumer and Producer which sends and receives Java Objects. Full Source I tried this: Producer: @Configuration public class KafkaProducerConfig { @Value(value = "${kafka.bootstrapAddress}") private String bootstrapAddress; @Bean public ProducerFactory<String, SaleRequestFactory> saleRequestFactoryProducerFactory() { Map<String, Object> configProps = new HashMap<>(); configProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapAddress); configProps.put

问题 We have a spring boot application which consumes messages from IBM MQ does some transformation and publishes the result to a Kafka topic. We use https://spring.io/projects/spring-kafka for this. I am aware that Kafka does not supports XA; however, in the documentation I found some inputs about using a ChainedKafkaTransactionManager to chain multiple transaction managers and synchronise the transactions. The same documentation also provides an example about how to synchronise Kafka and

问题 In a spring project i used Kafka and now I want to make a method which takes "TopicName" and "GroupeId" as parameters and calculate the difference between "Lastoffsets of the topic partitions" and the "offsets consumed by the group" for the lastOffsets i get it now i need to get the consumed offsets to calculate the difference public ResponseEntity<Offsets> deltaoffsets (@RequestParam( name = "groupId") String groupId, @RequestParam( name = "topic") String topic) { Map<String,Object>

问题 In a spring project i used Kafka and now I want to make a method which takes "TopicName" and "GroupeId" as parameters and calculate the difference between "Lastoffsets of the topic partitions" and the "offsets consumed by the group" for the lastOffsets i get it now i need to get the consumed offsets to calculate the difference public ResponseEntity<Offsets> deltaoffsets (@RequestParam( name = "groupId") String groupId, @RequestParam( name = "topic") String topic) { Map<String,Object>

问题 In a spring project i used Kafka and now I want to make a method which takes "TopicName" and "GroupeId" as parameters and calculate the difference between "Lastoffsets of the topic partitions" and the "offsets consumed by the group" for the lastOffsets i get it now i need to get the consumed offsets to calculate the difference public ResponseEntity<Offsets> deltaoffsets (@RequestParam( name = "groupId") String groupId, @RequestParam( name = "topic") String topic) { Map<String,Object>