问题
This is an extension of a previous question I asked: Is it possible to change the metadata of a column that is on a partitioned table in Hive?
Question: Is it possible to change an existing column's metadata on an EXTERNAL table that is defined by an AVRO schema file?
I need to change the column metadata on a table that is both partitioned and stored as EXTERNAL. The column itself is not the partitioning column. The metadata is stored in a separate AVRO file. I can confirm that the updated metadata is in the AVRO file, but not in the individual partitioned file.
In order to change the metadata in the partitioned files, I ran the ALTER TABLE TableName CHANGE COLUMN ColumnName ColumnName BIGINT CASCADE;
command, but I get the following error:
Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Not allowed to alter schema of Avro stored table having external schema. Consider removing avro.schema.literal or avro.schema.url from table properties.
I went ahead and successfully ran the following command:
ALTER TABLE TableName UNSET TBLPROPERTIES ('avro.schema.url');
I then successfully ran the CHANGE COLUMN command:
ALTER TABLE TableName CHANGE COLUMN ColumnName ColumnName BIGINT CASCADE;
When I perform a SELECT
on the table, I still get an error that indicates the metadata does not match between what is in the metastore versus what is in the partition file. I confirmed this by opening the partition data file. So, it appears that the CASCADE option is not applying to the partition.
来源:https://stackoverflow.com/questions/58369559/is-it-possible-to-change-an-existing-columns-metadata-on-an-external-table-that