Creating dataframe with pyspark (in databricks) takes forever, while it is pretty fast to do the same in pandas

Question

In short, my code loads data into lists and then I create dataframe from these lists I have no problems creating dataframe from lists in my local environments with pandas: