Reading from compressed files in Dataflow

后端 未结 4 1745
半阙折子戏
半阙折子戏 2021-01-14 14:07

Is there a way (or any kind of hack) to read input data from compressed files? My input consists of a few hundreds of files, which are produced as compressed with gzip and u

4条回答
  •  臣服心动
    2021-01-14 14:22

    I also found that for files that reside in the cloud store, setting the content type and content encoding appears to "just work" without the need for a workaround.

    Specifically - I run

    gsutil -m setmeta -h "Content-Encoding:gzip" -h "Content-Type:text/plain" 
    

提交回复
热议问题