spark-ec2 not recognized when lauching cluster on windows 8.1

会有一股神秘感。 提交于 2019-12-11 02:07:13

问题


I'm a complete beginner on spark. I'm trying to run spark on Amazon EC2, but my system does not recognize "spark-ec2" or "./spark-ec2". It says "spark-ec2" is not recognized as an internal or external command.

I followed the instruction here to launch a cluster. I would like to use Scala, how do I make it work?


回答1:


Add PYTHON PATH environment variable with boto. PYTHONPATH="${SPARK_EC2_DIR}/third_party/boto-2.4.1.zip/boto-2.4.1:$PYTHONPATH" And execute the python script




回答2:


In order to run the Spark-EC2 script on Windows you need Cygwin and Python. If you don't want to install these programs, you can use the dockerized version of the script (https://github.com/edrevo/spark-ec2-docker), which only depends on Docker.



来源:https://stackoverflow.com/questions/29787016/spark-ec2-not-recognized-when-lauching-cluster-on-windows-8-1

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!