I found at http://blog.madhukaraphatak.com/spark-3-introduction-part-3/ the following example to read all files from a nested directory:
val recursiveDf = sp