How to convert json to pyspark dataframe (faster implementation) [duplicate]

问题

This question already has answers here:

reading json file in pyspark (3 answers)

Closed 2 years ago.

I have json data in form of {'abc':1, 'def':2, 'ghi':3} How to convert it into pyspark dataframe in python?

回答1:

import json
j = {'abc':1, 'def':2, 'ghi':3}
a=[json.dumps(j)]
jsonRDD = sc.parallelize(a)
df = spark.read.json(jsonRDD)

>>> df.show()
+---+---+---+
|abc|def|ghi|
+---+---+---+
|  1|  2|  3|
+---+---+---+

来源：https://stackoverflow.com/questions/44456076/how-to-convert-json-to-pyspark-dataframe-faster-implementation