Multiple aggregations in Spark Structured Streaming

前端 未结 8 1729
难免孤独
难免孤独 2020-12-09 04:09

I would like to do multiple aggregations in Spark Structured Streaming.

Something like this:

  • Read a stream of input files (from a folder)
  • Per
8条回答
  •  时光说笑
    2020-12-09 04:39

    This is not supported, but there are other ways also. Like performing single aggregation and saving it to kafka. Read it from kafka and apply aggregation again. This has worked for me.

提交回复
热议问题