How do I make sure gzipped parquet files are the correct size on hdfs?

前端 未结 0 1467
再見小時候
再見小時候 2020-11-27 05:04

I\'m writing thousands of data frames as gzip parquet files with a target size of 512mb. However, hitting 512mb is hard. With a number of heuristics, i.e. compression ratios

相关标签:
回答
  • 消灭零回复
提交回复
热议问题