How to run a Spark Java program

前端 未结 4 828
一个人的身影
一个人的身影 2021-01-29 21:01

I have written a Java program for Spark. But how to run and compile it from Unix command line. Do I have to include any jar while compiling for running

4条回答
  •  一生所求
    2021-01-29 21:47

    Aditionally to the selected answer, if you want to connect to an external standalone Spark instance:

    SparkConf conf =
    new SparkConf()
         .setAppName("Simple Application")
         .setMaster("spark://10.3.50.139:7077");
    
    JavaSparkContext sc = new JavaSparkContext(conf);
    

    Here you can find more "master" configuration depending on where Spark is running: http://spark.apache.org/docs/latest/submitting-applications.html#master-urls

提交回复
热议问题