kafka-join

Kafka Streams join by key with complex condition

若如初见. 提交于 2020-06-29 06:51:56
问题 I'm trying to join KStream with GlobalKTable by key, but with specific logic. StreamsBuilder builder = new StreamsBuilder(); KStream<String, Integer> stream = builder.stream(inputTopic1); // key = "ABC" GlobalKTable<String, Integer> table = builder.globalTable(inputTopic2); // key = "ABC" stream.join(table, // join first by "ABC" = "ABC", then by "AB" = "AB", then by "A" = "A" (key, value) -> key, (valueLeft, valueRigth) -> {/* identify by which condition the join was performed */}); For

Embedded Kafka: KTable+KTable leftJoin produces duplicate records

五迷三道 提交于 2019-12-02 08:16:10
问题 I come seeking knowledge of the arcane. First, I have two pairs of topics, with one topic in each pair feeding into the other topic. Two KTables are being formed by the latter topics, which are used in a KTable+KTable leftJoin. Problem is, the leftJoin producing THREE records when I produce a single record to either KTable. I would expect two records in the form (A-null, A-B) but instead I get (A-null, A-B, A-null). I have confirmed that the KTables are receiving a single record each. I have