Spark 1.2.1 standalone cluster mode spark-submit is not working

前端 未结 2 605
醉酒成梦
醉酒成梦 2020-12-21 16:32

I have 3 node spark cluster

node1 , node2 and node 3

I running below command on node 1 for deploying driver

/usr/local/spark-1.2.1-bin-hadoop2.4/b

相关标签:
2条回答
  • 2020-12-21 16:57

    Cluster mode is not supported in EC2 1.2 instances where it creates a standalone cluster. Hence you can try removing

    --deploy-mode cluster --supervise 
    
    0 讨论(0)
  • 2020-12-21 17:05

    after spending lot more time.i got the answer.i did below changes

    1. remove entry of SPARK_LOCAL_IP and SPARK_MASTER_IP
    2. add host name and private ip address of each other nodes in etc/hosts.
    3. use --deploy-mode cluster --supervise

    thats all and it works perfectly with fully HA components(Master,Slaves and Driver)

    Thanks

    0 讨论(0)
提交回复
热议问题