I am facing an issue of how to split a multi-value column, i.e. List[String], into separate rows.
List[String]
The initial dataset has following types: Dataset
Dataset
You can use explode:
explode
df.withColumn("property", explode($"property"))
Example:
val df = Seq((1, List("a", "b"))).toDF("A", "B") // df: org.apache.spark.sql.DataFrame = [A: int, B: array] df.withColumn("B", explode($"B")).show +---+---+ | A| B| +---+---+ | 1| a| | 1| b| +---+---+