partition column in hive

Question

I have to partition the table in hive with a column which is also part of the table.

For eg:

Table: employee

Co

Answer 1

Here is how partition table in hive works: 1) the partition column data/value is not stored in the actual data file in warehouse instead it is stored in the hive meta store.

2) so you should not have the partition column data in the data files in the hive warehouse directory.

for your problem these should be the steps .

1)

CREATE TABLE employee (employeeId INT, employeeName STRING ) PARTITIONED BY (employeeSalary INT) stored as ;

This will create an entry in the hive metastore that you have created a table with 2 columns employeeId INT, employeeName STRING and it is having one partition column employeeSalary INT.

2) create a temp table lets say emp_temp.

CREATE TABLE emp_temp (employeeId INT, employeeName STRING,employeeSalary INT ) stored as text; 

i am assuming your input files are in text format.

3) copy all you file in the warehouse location of emp_temp table OR run the following query( i am asuming you have all you data files in the ./example/files folder.)

LOAD DATA LOCAL INPATH './examples/files/*.txt' OVERWRITE INTO TABLE emp_temp.

4) now run the following hql ( this will create partitions dynamically for you)

 INSERT OVERWRITE TABLE employee partition(employeeSalary) 
   SELECT employeeId , employeeName , employeeSalary 
   from emp_temp

Thanks, aditya