How to use pivot to generate a single-row matrix?

Question

I need to pivot the following two-column dataframe to one-row one (long to wide).

+--------+-----+
|   udate|   cc|
+--------+-----+
|20090622|  458|
|20090         


        

Answer 1

tl;dr In Spark 2.4.0 it simply boils down to using groupBy alone.

val solution = d.groupBy().pivot("udate").agg(first("cc"))
scala> solution.show
+--------+--------+--------+--------+--------+
|20090622|20090624|20090626|20090629|20090914|
+--------+--------+--------+--------+--------+
|     458|   31068|     151|     148|     453|
+--------+--------+--------+--------+--------+

If you really need the first column with the names just use withColumn and you're done.

val betterSolution = solution.select(lit("cc") as "udate", $"*")
scala> betterSolution.show
+-----+--------+--------+--------+--------+--------+
|udate|20090622|20090624|20090626|20090629|20090914|
+-----+--------+--------+--------+--------+--------+
|   cc|     458|   31068|     151|     148|     453|
+-----+--------+--------+--------+--------+--------+