Apache Spark type mismatch of the same type (String)

问题

EDIT: Answer: It was a JAR file that created a conflict! The related post is: Must include log4J, but it is causing errors in Apache Spark shell. How to avoid errors?

Doing the following:

val numOfProcessors:Int = 2
val filePath:java.lang.String = "s3n://somefile.csv"
var rdd:org.apache.spark.rdd.RDD[java.lang.String] = sc.textFile(filePath, numOfProcessors)

I get

    error: type mismatch;
 found   : org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.RDD[String]
 required: org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.RDD[String]
       var rdd:org.apache.spark.rdd.RDD[java.lang.String] = sc.textFile(filePath, numOfProcessors)

---

EDIT: Second case

val numOfProcessors = 2
val filePath = "s3n://somefile.csv"
var rdd = sc.textFile(filePath, numOfProcessors) //OK!

def doStuff(rdd: RDD[String]): RDD[String] = {rdd}

doStuff(rdd)

I get:

error: type mismatch;
 found   : org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.RDD[String]
 required: org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.RDD[String]
              doStuff(rdd)
                      ^

No comment...

Any ideas why I get this error ?

回答1:

The problem was a JAR file that created a conflict.

来源：https://stackoverflow.com/questions/29371592/apache-spark-type-mismatch-of-the-same-type-string