Spark job fails: storage.DiskBlockObjectWriter: Uncaught exception while reverting partial writes to file

Question

I have a Spark (1.4.1) application, running on Yarn, that fails with the following executor log entry:

16/07/21 23:09:08 ERROR executor.CoarseGrainedExecutor         


        

Answer 1

The reason caused by temp shuffle file is deleted. There are many reasons, for one which I met is because the other executor was killed by Yarn. After the executor killed, a SHUT_DOWN signal will be sent to other executors, then the ShutdownHookManager will delete all the temp files which have registered to ShutdownHookManager. That's why you see the error. So you maybe need to check whether there are any ShutdownHookManager called log.