How to convert empty arrays to nulls?

Question

I have below dataframe and i need to convert empty arrays to null.

+----+---------+-----------+
|  id|count(AS)|count(asdr)|
+----+---------+-----------+
|11         


        

Answer 1

you can do it with selectExpr:

df_filled = df.selectExpr(
    "id",
    "if(size(column1)<=0, null, column1)",
    "if(size(column2)<=0, null, column2)",
    ...
)