Querying Spark SQL DataFrame with complex types

Question

How Can I query an RDD with complex types such as maps/arrays? for example, when I was writing this test code:

case class Test(name: String, map: Map[String,         


        

Answer 1

here was what I did and it worked

case class Test(name: String, m: Map[String, String])
val map = Map("hello" -> "world", "hey" -> "there")
val map2 = Map("hello" -> "people", "hey" -> "you")
val rdd = sc.parallelize(Array(Test("first", map), Test("second", map2)))
val rffffdf = rdd.toDF
rffffdf.registerTempTable("mytable")
sqlContext.sql("select m.hello from mytable").show

Results

+------+
| hello|
+------+
| world|
|people|
+------+