Apache Beam Wait.on JdbcIO.write with unbounded PCollection issue

与世无争的帅哥 提交于 2021-01-29 13:14:05

问题


I am trying use the below scenario with unbounded pCollection datasource(PubSub). https://issues.apache.org/jira/browse/BEAM-6732

I am able to write to DB1. DB2 is having a Wait.on DB1 (PCollection .withResults). But unfortunately DB2 is not getting updated.

When I change the source to a bounded dummy PCollection, it works. Any input is appreciated.


回答1:


As I mentioned on Jira, do you have any Windows using in your unbounded pipeline? The writing to another database starts only after the respective window has been fully written to the first database. Otherwise, it will wait forever.



来源:https://stackoverflow.com/questions/58464172/apache-beam-wait-on-jdbcio-write-with-unbounded-pcollection-issue

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!