How to find pyspark dataframe memory usage?

后端 未结 4 421
情深已故
情深已故 2021-02-03 12:29

For python dataframe, info() function provides memory usage. Is there any equivalent in pyspark ? Thanks

4条回答
  •  别跟我提以往
    2021-02-03 13:04

    How about below? It's in KB, X100 to get the estimated real size.

    df.sample(fraction = 0.01).cache().count()
    

提交回复
热议问题