NoSuchMethodError from spark-cassandra-connector with assembled jar

Question

I\'m fairly new to Scala and am trying to build a Spark job. I\'ve built ajob that contains the DataStax connector and assembled it into a fat jar. When I try to execute it

Answer 1

Cassandra connector for Spark 1.6 is still in development and not released yet.

For Integrating Cassandra with Spark you need at-least following dependencies: -

1. Spark-Cassandra connector - Download appropriate version from here
2. Cassandra Core driver - Download appropriate version from here
3. Spark-Cassandra Java library - Download appropriate version from here
4. Other Dependent Jars - jodatime , jodatime-convert, jsr166

The mapping of appropriate version of Cassandra Libraries and Spark are mentioned here

Apparently the Cassandra connector for Spark 1.5 is also is in development and you may see some compatibility issues. The most stable release of Cassandra connector is for Spark 1.4 which requires following Jar Files: -

1. Spark-Cassandra connector
2. Cassandra Core driver
3. Spark-Cassandra Java library
4. Other Dependent Jars - jodatime , jodatime-convert, jsr166

Needless to mention that all these jar files should be configured and available to executors.