How to convert a column from hex string to long?

Question

I have a DataFrame with Icao column with hex codes that I would like to convert to Long datatype. How could I do this in Spark SQL?

Answer 1

You can use java hex to Long converter

java.lang.Long.parseLong(hex.trim(), 16)

All you need is to define a udf function as below

import org.apache.spark.sql.functions.udf
def hexToLong = udf((hex: String) => java.lang.Long.parseLong(hex.trim(), 16))

And call the udf function using .withColumn api

df.withColumn("Icao", hexToLong($"Icao")).show(false)