Unable to run a basic GraphFrames example

Question

Trying to run a simple GraphFrame example using pyspark.

spark version : 2.0

graphframe version : 0.2.0

I am able to import graphframes in Jupyter:

Answer 1

The simplest way is to start jupyter with pyspark and graphframes is to start jupyter out from pyspark with the respective packages

Just open your terminal and set the two environment variables and start pyspark with the graphframes package

export PYSPARK_DRIVER_PYTHON=jupyter
export PYSPARK_DRIVER_PYTHON_OPTS=notebook
pyspark --packages graphframes:graphframes:0.6.0-spark2.3-s_2.11

the advantage of this is also that if you later on want to run your code via spark-submit you can use the same start command