I want to convert string variable below to dataframe on spark.
val jsonStr = \"{ \"metadata\": { \"key\": 84896, \"value\": 54 }}\"
I know
you can now directly read json from Dataset[String]: https://spark.apache.org/docs/latest/sql-data-sources-json.html
val otherPeopleDataset = spark.createDataset(
"""{"name":"Yin","address":{"city":"Columbus","state":"Ohio"}}""" :: Nil)
val otherPeople = spark.read.json(otherPeopleDataset)
otherPeople.show()
// +---------------+----+
// | address|name|
// +---------------+----+
// |[Columbus,Ohio]| Yin|
// +---------------+----+