I am using the @pandas_udf to create the prediction job with fbprophet. My spark conf setting:
@pandas_udf
fbprophet
export PYSPARK_PYTHON=python3 pyspark