How does unbound table work in spark structured streaming

前端 未结 1 1758
难免孤独
难免孤独 2021-01-23 02:55

Take word count for example, when the application startup and long runs, and receive a word \"Spark\", then in the result table, there is a row (Spark,1),

A

相关标签:
1条回答
  • 2021-01-23 03:29

    To avoid this huge amount of data in memory spark structured streaming uses watermarks. The main idea is to store in memory only data within specific time window. All the data outside this window are stored in file system. You can read about watermarks here or here

    0 讨论(0)
提交回复
热议问题