Write and run pyspark in IntelliJ IDEA

Question

i am trying to work with Pyspark in IntelliJ but i cannot figure out how to correctly install it/setup the project. I can work with Python in IntelliJ and I can use the pyspark

Answer 1

Set the env path for (SPARK_HOME and PYTHONPATH) in your program run/debug configuration.

For instance:

SPARK_HOME=/Users//javalibs/spark-1.5.0-bin-hadoop2.4/python/
PYTHON_PATH=/Users//javalibs/spark-1.5.0-bin-hadoop2.4/python/pyspark

See attached snapshot in IntelliJ Idea