How to add an integer unique id to query results - __efficiently__?

Question

Given a query, select * from ... (that might be part of CTAS statement)

The goal is to add an additional column, ID, where ID is a

Answer 1

Check this solution from Manoj Kumar: https://github.com/manojkumarvohra/hive-hilo

• A stateful UDF is created which maintains a HI/LO counters to increment the sequences.
• The HI value is maintained as distribute atomic long in zookeeper.
• The HI value is incremented & fetched for every n LO (default 200) iterations.
• The UDF supports a single String argument which is the sequence name used to maintain zNodes in zookeeper.

Usage:

FunctionName( sequenceName, lowvalue[optional], seedvalue[optional])