I have a pyspark dataframe with the following schema
+----------+-------------------+-----------------------------------+------------------+ | date|