Is there better way to display entire Spark SQL DataFrame?

Question

I would like to display the entire Apache Spark SQL DataFrame with the Scala API. I can use the show() method:

myDataFrame.show(Int.MaxValue)
         


        

Answer 1

Nothing more succinct than that, but if you want to avoid the Int.MaxValue, then you could use a collect and process it, or foreach. But, for a tabular format without much manual code, show is the best you can do.