How to find common elements among two array columns?

Question

I have two comma-separated string columns (sourceAuthors and targetAuthors).

val df = Seq(
  (\"Author1,Author2,Author3\",\"Author2,Aut         


        

Answer 1

That error means that your udf is returning unit ( no return at all, as void un Java )

Try this. You are applying the intersect over the original s1 and S2 rather than over the splitted ones.

def myUDF = udf((s1: String, s2: String) =>{

  val splitted1 = s1.split(",")


  val splitted2= s2.split(",")


splitted1.intersect(splitted2).length

} )