spark submit throws an error java.lang.ClassNotFoundException: scala.runtime.java8.JFunction2$mcIII$sp

Question

I wrote a code for word count but when I tried running it from CMD in windows using below command it throws an exception.

spark-submit --class com.sample.WordCou         


        

Answer 1

I had the same issue, and solved it by changing the version of Scala I use during development to match the version Spark came with.

When I start Spark with ./spark-shell, it says Using Scala version 2.11.12, So I changed the Scala version in the build.sbt from 2.12.8 to 2.11.12 and everything worked. I use Spark version 2.4.3.