Convert PySpark dataframe column from list to string

Question

I have this PySpark dataframe

+-----------+--------------------+
|uuid       |   test_123         |    
+-----------+--------------------+
|      1    |[test         


        

Answer 1

While you can use a UserDefinedFunction it is very inefficient. Instead it is better to use concat_ws function:

from pyspark.sql.functions import concat_ws

df.withColumn("test_123", concat_ws(",", "test_123")).show()

+----+----------------+
|uuid|        test_123|
+----+----------------+
|   1|test,test2,test3|
|   2|test4,test,test6|
|   3|test6,test9,t55o|
+----+----------------+