Why does Spark application fail with “Exception in thread ”main“ java.lang.NoClassDefFoundError: …StringDeserializer”?

Question

I am developing a Spark application that listens to a Kafka stream using Spark and Java.

I use kafka_2.10-0.10.2.1.

I have set various parameters for Kafka prope

Answer 1

spark-streaming_2.10

This is dependent upon Scala 2.10

Your other dependencies are using Scala 2.11

Upgrading the version is the correct solution for the current error.

And make sure that within streaming-kafka-0-10, this matches the version of Kafka you're running

Application is compiling fine but when I am trying to submit the spark job, its showing error: Exception in thread "main" java.lang.NoClassDefFoundError:

By default, maven does not include dependency jars when it builds a target