How to use Databricks Job Spark Configuration spark_conf?

问题

I have a sample Spark Code where I am trying to access the Values for tables from the Spark Configurations provided by spark_conf Option by using the typeSafe application.conf and Spark Conf in the Databricks UI. The code I am using is below,

When I hit the Run Button in the Databricks UI, the job is finishing successfully, but the println function is printing dummyValue instead of ThisIsTableAOne,ThisIsTableBOne...

I can see from the Spark UI that, the Configurations for TableNames are being passed to the Spark job, but these values are not getting reflected in the Code.

    try {

    val inputConfig = AppConfig.getConfig("input")
    val outputConfig = AppConfig.getConfig("output")

    val tableA = inputConfig.getString("tableA")
    val tableB = inputConfig.getString("tableB")
    val tableC = outputConfig.getString("tableC")

    println(
    s"""
    |$tableA
    |$tableB
    |$tableC
    |""".stripMargin)

    val userDataInTable = sparkSession.createDataFrame(Seq(
    (1, "dummy", "dummy", "dummy")

    )).toDF("id", "col2", "col3", "col4")

    userDataInTable.show(false)

    println("Completed Entry ")


    } catch {
    case e: Exception =>
    sparkSession.stop()
    e.printStackTrace
    }
    //application.conf contains below text,

    spark.tableNameA="dummyValue"
    spark.tableNameB="dummyValue"
    spark.tableNameC="dummyValue"
    input{
    tableA=${spark.tableNameA}
    tableB=${spark.tableNameB}
    }
    output{
    tableC=${spark.tableNameC}
    }

    //AppConfig
    val appConfig = ConfigFactory.load("application.conf")
    def getConfig(moduleName: String): Config = {
    val config = appConfig.getConfig(moduleName)
    config
    }

来源：https://stackoverflow.com/questions/62094327/how-to-use-databricks-job-spark-configuration-spark-conf