问题
I have a spark application that I am trying to execute on windows/Unix using below command
spark-submit --class com.myorg.dataquality.DataVerification --master local[*] C:\Users\workspace\dataQuality\target\data-quality-framework-0.0.1-SNAPSHOT.jar
The job is getting terminated immediately after submission. However it is working perfect in eclipse.
C:\Users>spark-submit --class com.myorg.dataquality.DataVerification --master local[*] C:\Users\workspace\dataQuality\target\data-quality-framework-0.0.1-SNAPSHOT.jar
19/09/04 19:33:19 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
log4j:WARN No appenders could be found for logger (org.apache.spark.deploy.SparkSubmit$$anon$2).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
C:\Users>
What could be the issue with it:
Spark Version 2.4.4
Scala Version 2.11.11
I am able to run the job using eclipse, however i am unable to execute it on windows/Linux standalone cluster. If I try spark-shell
it is initiating properly.
来源:https://stackoverflow.com/questions/57790353/spark-submit-jobs-fails-with-no-error-in-local-mode