Spark Unable to find JDBC Driver

Question

So I\'ve been using sbt with assembly to package all my dependencies into a single jar for my spark jobs. I\'ve got several jobs where I was using c3p0 to setu

Answer 1

With spark 2.2.0, problem was corrected for me by adding extra class path information for SparkSession session in python script :

    spark = SparkSession \
        .builder \
        .appName("Python Spark SQL basic example") \
        .config("spark.driver.extraClassPath", "/path/to/jdbc/driver/postgresql-42.1.4.jar") \
        .getOrCreate()

See official documentation https://spark.apache.org/docs/latest/configuration.html

In my case, spark is not launched from cli command, but from django framework https://www.djangoproject.com/