I\'m trying to get the path to spark.worker.dir for the current sparkcontext.
spark.worker.dir
sparkcontext
If I explicitly set it as a config param, I can read
config param
Not sure if you can get all the default settings easily, but specifically for the worker dir, it's quite straigt-forward:
from pyspark import SparkFiles print SparkFiles.getRootDirectory()