How to read and write Map from/to parquet file in Java or Scala?

前端 未结 3 899
一整个雨季
一整个雨季 2021-01-04 06:58

Looking for a concise example on how to read and write Map from/to parquet file in Java or Scala?

Here is expected structure, usin

3条回答
  •  栀梦
    栀梦 (楼主)
    2021-01-04 07:20

    Apache Drill is your answer!

    Convert to parquet : You can use the CTAS(create table as) feature in drill. By default drill creates a folder with parquet files after executing the below query. You can substitute any query and drill writes the output of you query into parquet files

    create table file_parquet as select * from dfs.`/data/file.json`;
    

    Convert from parquet : We also use the CTAS feature here, however we request drill to use a different format for writing the output

    alter session set `store.format`='json';
    create table file_json as select * from dfs.`/data/file.parquet`;
    

    Refer to http://drill.apache.org/docs/create-table-as-ctas-command/ for more information

提交回复
热议问题