Does Dataflow templating supports template input for BigQuery sink options?

前端 未结 1 1675
广开言路
广开言路 2021-01-23 15:12

As I have a working static Dataflow running, I\'d like to create a template from this one to let me easily reuse the Dataflow without any command line typing.

Following

相关标签:
1条回答
  • 2021-01-23 15:38

    Python currently only supports ValueProvider options for FileBasedSource IOs. You can see that by clicking on the Python tab at the link you mentioned: https://cloud.google.com/dataflow/docs/templates/creating-templates

    under the "Pipeline I/O and runtime parameters" section.

    Unlike what happens in Java, BigQuery in Python does not use a custom source. In other words, it is not fully implemented in the SDK but also contains parts in the backend (and it is therefore a "native source"). Only custom sources can use templates. There are plans to have BigQuery added as custom source: issues.apache.org/jira/browse/BEAM-1440

    0 讨论(0)
提交回复
热议问题