How to extract a single (column/row) value from a dataframe using PySpark?

问题

Here's my spark code. It works fine and returns 2517. All I want to do is to print "2517 degrees"...but I'm not sure how to extract that 2517 into a variable. I can only display the dataframe but not extract values from it. Sounds super easy but unfortunately I'm stuck! Any help will be appreciated. Thanks!

df = sqlContext.read.format("csv").option("header", "true").option("inferSchema", "true").option("delimiter", "\t").load("dbfs:/databricks-datasets/power-plant/data")

df.createOrReplaceTempView("MyTable")

df = spark.sql("SELECT COUNT (DISTINCT AP) FROM MyTable")

display(df)

回答1:

I think you're looking for collect. Something like this should get you the value:

df.collect()[0]['count(DISTINCT AP)']

assuming the column name is 'count(DISTINCT AP)'

来源：https://stackoverflow.com/questions/55406069/how-to-extract-a-single-column-row-value-from-a-dataframe-using-pyspark