How to spark-submit with main class in jar?

前端 未结 4 1981
慢半拍i
慢半拍i 2021-02-10 15:02

There are a ton of questions about ClassNotFoundException but I haven\'t seen any (yet) that fit this specific case. I am attempting to run the following command:

4条回答
  •  太阳男子
    2021-02-10 15:34

    Why don't you use the path to the jar file so spark-submit (as any other command line tool) could find and use it?

    Given the path out/artifacts/TimeSeriesFilter_jar/scala-ts.jar I'd use the following:

    spark-submit --class com.stronghold.HelloWorld out/artifacts/TimeSeriesFilter_jar/scala-ts.jar
    

    Please note that you should be in the project's main directory which seems to be /home/[USER]/projects/scala_ts.

    Please also note that I removed --master local[*] since that's the default master URL spark-submit uses.

提交回复
热议问题