Zeppelin 0.7.2 version does not support spark 2.2.0

你说的曾经没有我的故事 提交于 2019-12-24 03:15:55

问题


How to downgrade the spark version? What could be the other solutions? I have to connect my hive tables to spark using spark session. But the spark version is not supported by zeppelin.


回答1:


Here are 2 reasons.

[1] Zeppelin 0.7.2 marked spark 2.2+ as the unsupported version.

https://github.com/apache/zeppelin/blob/v0.7.2/spark/src/main/java/org/apache/zeppelin/spark/SparkVersion.java#L40

public static final SparkVersion UNSUPPORTED_FUTURE_VERSION = SPARK_2_2_0;

[2] Even if you change the const and build again, you might fail to run Zeppelin 0.7.2 with Spark 2.2

  • https://spark.apache.org/releases/spark-release-2-2-0.html

Spark dropped support for Java 7 and Zeppelin 0.7.2 was built with JDK 7. So you need to rebuild it with JDK 8.

One work around you can use is, specifying JAVA_HOME in the spark interpreter (for 2.2) as commented here

  • https://github.com/apache/zeppelin/pull/2486#issuecomment-314954959

That works because only spark interpreter (for 2.2) requires java 8 and Zeppelin doesn't need it.

In short

  1. Modify here and rebuild if you want to spark 2.2 on Zeppelin branch-0.7
  2. And Use JDK 8 for the spark interpreter.



回答2:


Zeppelin 0.7.2 version supports spark 2.1.0.



来源:https://stackoverflow.com/questions/45789231/zeppelin-0-7-2-version-does-not-support-spark-2-2-0

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!