I like to test some simple pyspark logic locally before running it on AWS. I have a script that works fine until today:
from pyspark.sql import SparkSession