I wanted check with you for any possibility to parallelize the below steps in spark? currently I have written a pyspark function , using for loop over each row of dataframe and