Convert between spark.SQL DataFrame and pandas DataFrame [duplicate]

Answer 1

Try:

spark_df.toPandas()

toPandas()

Returns the contents of this DataFrame as Pandas pandas.DataFrame.

This is only available if Pandas is installed and available.

And if you want the oposite:

spark_df = createDataFrame(pandas_df)