Unique Key generation in Hive/Hadoop

Question

While selecting a set of records from a big data hive table, a unique key needs to be created for each record. In a sequential mode of operation , it is easy to generate unique

Answer 1

SELECT T.*, ROW_NUMBER () OVER (ORDER BY T.C1) AS SEQ_NBR 
FROM TABLE T

Here C1 is any numeric column in T. This will generate a unique number for each record while selecting from table T, starting from 1. If this is one time activity then solution is fine.

In case you need to repeat this process every day and insert this data into table T2 and generate unique id then you can try below method.

SELECT T.*, ROW_NUMBER () OVER (ORDER BY T.C1)+ SEQ_T2  AS SEQ_NBR 
FROM TABLE T, (SELECT MAX(SEQ) AS SEQ_T2 FROM TABLE T2)

Hope it helps !!