Write data from pyspark to azure blob?

问题

I want to write dataframe from pyspark to azure blob? Any suggestions or code how to do it?

I have location and key of blob

enter image description here

回答1:

You could follow this tutorial to connector your spark dataframe with Azure Blob Storage.

Set connection info:

session.conf.set(
    "fs.azure.account.key.<storage-account-name>.blob.core.windows.net",
    "<your-storage-account-access-key>"
)

Then write data into blob storage:

sdf = session.write.parquet(
    "wasbs://<container-name>@<storage-account-name>.blob.core.windows.net/<prefix>"
)

Also,you could refer to this case:pyspark write to wasb blob storage container

来源：https://stackoverflow.com/questions/56983295/write-data-from-pyspark-to-azure-blob