spark build path is cross-compiled with an incompatible version of Scala (2.10.0)

Question

When i try to execute sparksql code in scala IDE im getting below error,Could anyone help me to sort out this please?

spark build path is cross-compiled wit         


        

Answer 1

In your project you are using jars built with different Scala versions; indeed from the log:

• Scala IDE uses Scala 2.11.7
• Apache Spark 1.5.2 built with Scala 2.10

You need to align the jars versions. Actually Spark 1.5.2 pre-build (downloadable from here) has built with Scala 2.10 for compatibility reasons (see here). On the web site there is a note:

Note: Scala 2.11 users should download the Spark source package and build with Scala 2.11 support.

In order to solve the issue, in your project use the same Scala version used to build Spark.

I suggest to try to switch to Scala 2.10 in your Scala IDE and it will solve the issue.

Eclipse + Maven

In my case, I'm using Eclipse with Scala IDE and Maven so I updated the Maven dependencies in this way:

<dependency>
    <groupId>org.scala-lang</groupId>
    <artifactId>scala-library</artifactId>
    <version>2.10.6</version>
</dependency>
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.10</artifactId>
    <version>1.5.2</version>
</dependency>

Then, I changed the Scala version in the IDE: Right click on the project -> Scala -> set Scala installation or Right Click on the Scala Library Container -> Properties and choose the Scala 2.10 as shown below

Answer 2

"-Xsource:2.11 -Ymacro-expand:none" solved my problem , just updated in right click project -> properties -> Scala Compiler , Additional Command Line Parameter , to fix this "build path is cross-compiled with an incompatible version of Scala (2.11.0). In case this report is mistaken, this check can be disabled in the compiler preference page."

Answer 3

Click right on your project, then select Properties. Select Scala Compiler Click "Use Project Setting" Select Scala Installation select your scala version, for example, 2.11.11.

Answer 4

I had same problem. How I solved it. Scala embedded Eclipse -> Project -> Right Click -> Project -> Scala Compiler -> Right side you will be getting list of Scala Installation-> Here choose Latest 2.10 bundle (dynamic).

I have tested and it worked fine for me.

• Binod Suman