Drop if all entries in a spark dataframe's specific column is null

Question

Using Pyspark, how can I select/keep all columns of a DataFrame which contain a non-null value; or equivalently remove all columns which contain no data.

Answer 1

Or just

from pyspark.sql.functions import col

for c in df.columns:
    if df.filter(col(c).isNotNull()).count() == 0:
      df = df.drop(c)