Emrfs file sync with s3 not working

Question

After running a spark job on an Amazon EMR cluster, I deleted the output files directly from s3 and tried to rerun the job again. I received the following error upon trying to w

Answer 1

It turned out that I needed to run

emrfs delete s3://bucket/folder

first before running sync. Running the above solved the issue.