Dynamic Partitioning + CREATE AS on HIVE

后端 未结 2 813
Happy的楠姐
Happy的楠姐 2021-02-05 16:28

I\'m trying to create a new table from another table with CREATE AS and dynamic Partitioning on HiveCLI. I\'m learning from Hive official wiki where there is this e

相关标签:
2条回答
  • 2021-02-05 17:03

    Since you already know the full schema of the target table, try creating it first and the populating it with a LOAD DATA command:

    SET hive.exec.dynamic.partition.mode=nonstrict;
    
    CREATE TABLE T (key int, value string) 
    PARTITIONED BY (ds string, hr int);
    
    INSERT OVERWRITE TABLE T PARTITION(ds, hr) 
    SELECT key, value, ds, hr+1 AS hr 
       FROM srcpart 
       WHERE ds is not null 
       And hr>10;
    

    Note: the set command is needed since you are performing a full dynamic partition insert.

    0 讨论(0)
  • 2021-02-05 17:10
    SET hive.exec.dynamic.partition.mode=nonstrict;
    
    CREATE TABLE T (key int, value string) 
    PARTITIONED BY (ds string, hr int);
    
    INSERT OVERWRITE TABLE T PARTITION(ds, hr) 
    SELECT key, value, ds, hr+1 AS hr 
    FROM srcpart 
    WHERE ds is not null 
          And hr>10;
    

    In the above code, instead of the Create statement use: CREATE TABLE T like srcpart;

    In case the partitioning is similar.

    0 讨论(0)
提交回复
热议问题