parquet.io.ParquetDecodingException: Can not read value at 0 in block -1 in file

前端 未结 6 679
独厮守ぢ
独厮守ぢ 2020-12-31 01:44

I have saved a remote DB table in Hive using saveAsTable method, now when i try to access the Hive table data using CLI command select * from table_name

6条回答
  •  有刺的猬
    2020-12-31 02:23

    One more way to catch possible discrepancy is to eyeball the difference in schemata of parquet files produced by both sources, say hive and spark. You can dump schema with parquet-tools (brew install parquet-tools for macos):

    λ $ parquet-tools schema /usr/local/Cellar/apache-drill/1.16.0/libexec/sample-data/nation.parquet
    message root {
      required int64 N_NATIONKEY;
      required binary N_NAME (UTF8);
      required int64 N_REGIONKEY;
      required binary N_COMMENT (UTF8);
    }
    

提交回复
热议问题