KafkaUtils class not found in Spark streaming

Question

I have just began with Spark Streaming and I am trying to build a sample application that counts words from a Kafka stream. Although it compiles with sbt package

Answer 1

You could also download the jar file and put it in the Spark lib folder, because it is not installed with Spark, instead of beating your head trying to bet SBT build.sbt to work.

http://central.maven.org/maven2/org/apache/spark/spark-streaming-kafka-0-10_2.10/2.1.1/spark-streaming-kafka-0-10_2.10-2.1.1.jar

copy it to:

/usr/local/spark/spark-2.1.0-bin-hadoop2.6/jars/

Answer 2

Using Spark 1.6 do the job for me without the hassle of handling so many external jars... Can get quite complicate to manage...

Answer 3

Added the Dependency externally, project-->properties--> java Build Path-->Libraries--> add External jars and add the required jar.

this solved my issue.