I was looking for a formula to optimize the spark.shuffle.partitions and came across this post It mentions spark.sql.shuffle.partitions = quotient (shuffle st
spark.shuffle.partitions
spark.sql.shuffle.partitions = quotient (shuffle st