Flink and Dynamic templates recognition

怎甘沉沦 提交于 2019-12-22 00:26:49

问题


We plan to use Flink CEP for processing a big amount of events according to some dynamic templates. The system must recognize chains of events (sometimes complicated chains with conditions and grouping). The templates will be created by user. In other words we have to create complicated templates without touching the code. Is it possible to use Apache Flink for solving this problem? Does Filnk support dynamic-templates?


回答1:


At the moment Flink's CEP library does not support this kind of dynamic rule adaption. However, there is no fundamental reason which makes it impossible to implement.

In fact, the changes should be rather straight forward. At the moment the workflow is the following: The user specifies an event pattern. This pattern is translated into a NFA representing the pattern (this happens on the client side). The NFA is then given to the CEP operator which is serialized and shipped to the cluster. On the cluster the CEP operator and the NFA are deserialized and then start processing events.

What one could add is a co-flat map operator which receives on one input channel the events and on the other input channel patterns. For each newly received pattern one either updates the existing NFA (this functionality is missing) or compiles a new one. In the latter case, one would apply incoming events to all stored NFAs.

That way, one could realize a CEP operator whose pattern can be dynamically updated.



来源:https://stackoverflow.com/questions/40018199/flink-and-dynamic-templates-recognition

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!