Build a SparkSession

前端 未结 1 1287
小鲜肉
小鲜肉 2021-01-29 08:22

I have a spark as interpreter in Zeppelin. I\'m using a Spark2.0, I built a Session: Create

相关标签:
1条回答
  • 2021-01-29 09:02

    In general you should not initialize SparkSession nor SparkContext in Zeppelin. Zeppelin notebooks are configured to create session for you, and their correct behavior depends on using provided objects.

    Initializing your SparkSession will break core Zeppelin functionalities, and multiple SparkContexts will break things completely in the worst case scenario.

    Is set spark.driver.allowMultipleContexts to False is best to do a tests ?

    You should never use spark.driver.allowMultipleContexts - it not supported, and doesn't guarantee correct results.

    0 讨论(0)
提交回复
热议问题