Spark scala remove columns containing only null values

Question

Is there a way to remove the columns of a spark dataFrame that contain only null values ? (I am using scala and Spark 1.6.2)

At the moment I am doing this:

Answer 1

If the dataframe is of reasonable size, I write it as json then reload it. The dynamic schema will ignore null columns and you'd have a lighter dataframe.

scala snippet:

originalDataFrame.write(tempJsonPath)
val lightDataFrame = spark.read.json(tempJsonPath)