filter the lines by two words Spark Streaming

问题

Is there a way to filter with one expression the lines containing a word "word1" or the other "word2" something like :

val res = lines.filter(line => line.contains("word1" or "word2"))

because this expression doesn't work.

Thank you in advance

回答1:

If line is a String optimal choice would regexp:

val pattern = "word1|word2".r

lines.filter(line => pattern.findFirstIn(line).isDefined)

otherwise (other sequence type) you can use Seq.exists:

lines.filter(line => Seq("foo", "bar").exists(s => line.contains(s)))

which takes a single which maps from element to boolean (here (String) ⇒ Boolean) and:

tests whether a predicate holds for at least one element of this iterable collection.

来源：https://stackoverflow.com/questions/36650184/filter-the-lines-by-two-words-spark-streaming