I have a Spark project that used to work lately .
the project gets a CSV , and add two fields to it then it outputs with saveasTextfile()
the content of
I was getting similar error, realized I had not created the environment variable HADOOP_HOME and Path variable was not updated with HADOOP_HOME\bin
I had the same stack trace appear for me, but A.HADDAD's solution didn't work. I solved it by updating my hadoop version (which was 2.6.0) to 2.7.1, as it was closer the hadoop version indicated by my spark instalation (spark-2.3.2-bin-hadoop2.7).
I Solved my question :
After that worked for me lately i have put (to run another spark project) in the hadoop home hadoop.dll
along side to winutils.exe
.
so i removed hadoop.dll
and now everything is fine