Python spark : How parellelize Spark Dataframe compute using spark in databricks

Question

I have a python code that compute dataframe in parellelize using the library: multiprocessing.pool

from multiprocessing.pool import ThreadPool a