Databricks - Creating permanent User Defined Functions (UDFs)

Question

I am able to create a UDF function and register to spark using spark.UDF method. However, this is per session only. How to register python UDF functions automatically when

Answer 1

This is not possible; this is not like UDFs in Hive.

Code the UDF as part of the package / program you submit or in the jar included in the Spark App, if using spark-submit.

However,

spark.udf.register.udf("...

is required to be done as well. This applies to Databrick notebooks, etc. The UDFs need to be re-registered per Spark Context/Session.