Failed to load class for data source: Libsvm in spark ML pyspark/scala

问题

When I try to import a libsvm file in pyspark/scala using "sqlContext.read.format("libsvm").load", I get the following error -

"Failed to load class for data source: Libsvm."

At the same time, if I use "MLUtils.loadLibSVMFile" it works perfectly fine. I need to use both Spark ML (to get class probabilities) and MLlib for an evaluation. Have attached the error screenshot.

This is a MapR cluster. Spark version 1.5.2 Error

回答1:

libsvm source format is available since version 1.6 of Spark.

回答2:

this is probably a version mismatch, the version of libsvm you have in your project may not have that particular API, can you verify the version of libsvm is loaded and then determine if that api is available?

来源：https://stackoverflow.com/questions/35157032/failed-to-load-class-for-data-source-libsvm-in-spark-ml-pyspark-scala