Is it possible for a Kafka stream to write output to two different topics?

只愿长相守 提交于 2020-02-05 04:58:04

问题


I need a stream to group values from topic A, send the grouped values to topic B, and send the sum of those grouped values over a tumbling time window to topic C. Is it possible to do that in Kafka? Or can a stream both read and write from one topic only?


回答1:


Yes. For example you can split a input stream and write to different topics:

KStream stream = ...
KStream[] splitStream = stream.branch(...);

splitStream[0].to("topic-1");
splitStream[1].to("topic-2");

You can also use a more dynamic approach via to(TopicNameExtractor).

You can also "broadcast" a stream to apply different logic. Reusing the same KStream variable ensures that each record is logically duplicated and processed through multiple parallel downstream operations:

KStream stream = ...

KStream filterdStream = stream.filter();
KTable count = stream.groupBy().count();


来源:https://stackoverflow.com/questions/51745967/is-it-possible-for-a-kafka-stream-to-write-output-to-two-different-topics

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!