Is there a difference in `BigQueryIO` when you use `fromTable` vs `fromQuery(“SELECT * …”)` in dataflow?

后端 未结 1 1090
爱一瞬间的悲伤
爱一瞬间的悲伤 2021-01-16 03:26

When you need to read all the data from one or more tables in bigquery in a dataflow job there are two approaches to it I would say. The first one is to use BigQueryIO

1条回答
  •  一向
    一向 (楼主)
    2021-01-16 04:23

    from is both cheaper and faster than fromQuery(SELECT * FROM ...).

    • from directly exports the table and exporting data is free for BigQuery.
    • fromQuery(SELECT * FROM ...) will first scan the entire table ($5/TB) and export the result.

    0 讨论(0)
提交回复
热议问题