Creating hive partitions for multiple months using one script

你离开我真会死。 提交于 2019-12-06 10:53:15

问题


I have data for 4 years. Like

'2011
 2012
 2013
 2014'

I have to run queries based on one month's data. So i am creating partitions as below.

'ALTER TABLE table1_2010Jan ADD PARTITION(year='2010', month='01', day='01')
 LOCATION 'path';
 ALTER TABLE table1_2010Jan ADD PARTITION(year='2010', month='01', day='02')
 LOCATION 'path';
 ALTER TABLE table1_2010Jan ADD PARTITION(year='2010', month='01', day='03')
 LOCATION 'path';'

I am creating individual partitions like above for every day of every month.

I want to know if we can write a script(any language) an run it one time to create these partitions for the all the days of a month for every year? Or is there a way to do this in hive itself.

Any suggestions please..


回答1:


For external table , hive can not do it. You can create shell/perl script to to do the same (using loop).

For hive managed table, you can do it easily using MSCK REPAIR command.

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-RecoverPartitions%28MSCKREPAIRTABLE%29

Only thing you have to ensure that the data is loaded is correct partition.



来源:https://stackoverflow.com/questions/23766449/creating-hive-partitions-for-multiple-months-using-one-script

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!