Hive: modify external table's location take too long

送分小仙女□ 提交于 2020-07-07 05:38:09

问题


Hive has two kinds of tables which are Managed and External Tables, for the difference, you can check Managed. VS External Tables.

Currently, to move external database from HDFS to Alluxio, I need to modify external table's location to alluxio://.

The statement is something like: alter table catalog_page set location "alluxio://node1:19998/user/root/tpcds/1000/catalog_returns"

According to my understanding, it should be a simple metastore modification,however, for some tables modification, it will spend dozens of minutes. The database itself contains about 1TB data btw.

Is there anyway for me to accelerate the table alter process? If no, why it's so slow? Any comment is welcomed, thanks.


回答1:


I found suggested way which is metatool under $HIVE_HOME/bin.

metatool -updateLocation <new-loc> <old-loc>      Update FS root location in the
                                          metastore to new location.Both
                                          new-loc and old-loc should be
                                          valid URIs with valid host names
                                          and schemes.When run with the
                                          dryRun option changes are
                                          displayed but are not persisted.
                                          When run with the
                                          serdepropKey/tablePropKey option
                                          updateLocation looks for the
                                          serde-prop-key/table-prop-key
                                          that is specified and updates
                                          its value if found.

By using this tool, the location modification is very fast. (maybe several seconds.)

Leave this thread here for anyone who might run into the same situation.



来源:https://stackoverflow.com/questions/57653843/hive-modify-external-tables-location-take-too-long

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!