Can we combine both and count and process time Trigger in Flink?

99封情书 提交于 2020-01-24 20:27:11

问题


I want to make the Windows completed after the count reached 100 or every 5 seconds for the tumbling process time ? That is to say when the elements reached 100, trigger the Windows computation, however if the elements don't reache 100, but the time elapsed 5 seconds, it also trigger the Windows computation, just as the combination of the below two triggers:

.countWindow(100)

.window(TumblingProcessingTimeWindows.of(Time.seconds(5)))


回答1:


There's no super simple way to do this with the current Flink API.

Your use case needs a combination of state (for counting), and a timer. You can either accomplish this with windows using a custom Trigger, or by using a ProcessFunction.

For the approach with windows plus a custom trigger, looking at the implementations of ProcessingTimeTrigger and CountTrigger will be helpful, as you basically want to blend the two.

ProcessFunction is a lower-level building block that combines managed state with timers, which is exactly what you need, so this is probably easier, especially if you already know how to work with Flink's managed state.

BTW, the online Flink training materials include slides and exercises for implementing custom Triggers and for using ProcessFunction.

Triggers: slides, exercise
ProcessFunction: slides, exercise



来源:https://stackoverflow.com/questions/44382329/can-we-combine-both-and-count-and-process-time-trigger-in-flink

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!