apache-kafka-streams

Kafka compare consecutive values for a key

拈花ヽ惹草 提交于 2020-08-23 07:53:19
问题 We are building an application to get data from sensors. The data is streamed to Kafka from where consumers will publish it to different data stores. Each data point will have multiple attributes representing the state of the sensor. In one of the consumers we want to publish the data to the data store only if the value has changed. for e.g. if there is temperature sensor which is polled for data every 10 secs we expect to receive data like ----------------------------------------------------

Kafka compare consecutive values for a key

你离开我真会死。 提交于 2020-08-23 07:51:16
问题 We are building an application to get data from sensors. The data is streamed to Kafka from where consumers will publish it to different data stores. Each data point will have multiple attributes representing the state of the sensor. In one of the consumers we want to publish the data to the data store only if the value has changed. for e.g. if there is temperature sensor which is polled for data every 10 secs we expect to receive data like ----------------------------------------------------

How to make Spring cloud stream Kafka streams binder retry processing a message if a failure occurs during the processing step?

依然范特西╮ 提交于 2020-08-10 02:01:07
问题 I am working on Kafka Streams using Spring Cloud Stream. In the message processing application, there may be a chance that it will produce an error. So the message should not be commited and retried again. My application method - @Bean public Function<KStream<Object, String>, KStream<String, Long>> process() { return (input) -> { KStream<Object, String> kt = input.flatMapValues(v -> Arrays.asList(v.toUpperCase().split("\\W+"))); KGroupedStream<String, String> kgt =kt.map((k, v) -> new