Apache Flink - Dataset api - Side outputs

被刻印的时光 ゝ 提交于 2020-03-25 03:16:50

问题


Does Flink supports Side Outputs feature in Dataset(Batch Api) ? If not, how to handle valid and invalid records when loading from file ?


回答1:


You can always do something like this:

DataSet<EventOrInvalidRecord> goodAndBadTogether = input.map(new CreateObjectIfPossible())
goodAndBadTogether.filter(new KeepOnlyGood())...
goodAndBadTogether.filter(new KeepOnlyBad())...

Another reasonable option in some cases is to go ahead and use the DataStream API, even if you don't have streaming sources.



来源:https://stackoverflow.com/questions/60014914/apache-flink-dataset-api-side-outputs

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!