Compare data in two RDD in spark
I am able to print data in two RDD with the below code. usersRDD.foreach(println) empRDD.foreach(println) I need to compare data in two RDDs. How can I iterate and compare field data in one RDD with field data in another RDD. Eg: iterate the records and check if name and age in userRDD has a matching record in empRDD , if no put in separate RDD. I tried with userRDD.substract(empRDD) but it was comparing all the fields. Sean Owen You need to key the data in each RDD so that there is something to join records on. Have a look at groupBy for example. Then you join the resulting RDDs. For each key