Can unix_timestamp() return unix time in milliseconds in Apache Spark?

Question

I\'m trying to get the unix time from a timestamp field in milliseconds (13 digits) but currently it returns in seconds (10 digits).

scala> var df = Seq(\"20         


        

Answer 1

Milliseconds hide in fraction part timestamp format

Try this:

df = df.withColumn("time_in_milliseconds", col("time").cast("double"))

You'll get something like 1484758800.792, where 792 it's milliseconds

At least it's works for me (Scala, Spark, Hive)