spark submit “Service 'Driver' could not bind on port” error

Question

I used the following command to run the spark java example of wordcount:-

time spark-submit --deploy-mode cluster --master spark://192.168.0.7:6066 --class o         


        

Answer 1

I had the same issue when trying to run the shell, and was able to get this working by setting the SPARK_LOCAL_IP environment variable. You can assign this from the command line when running the shell:

SPARK_LOCAL_IP=127.0.0.1 ./bin/spark-shell

For a more permanent solution, create a spark-env.sh file in the conf directory of your Spark root. Add the following line:

SPARK_LOCAL_IP=127.0.0.1

Give execute permissions to the script using chmod +x ./conf/spark-env.sh, and this will set this environment variable by default.