Unable to run a basic GraphFrames example

Question

Trying to run a simple GraphFrame example using pyspark.

spark version : 2.0

graphframe version : 0.2.0

I am able to import graphframes in Jupyter:

Answer 1

Follow-up on @Gilles Essoki solution. Make sure you have the right Spark version and Scala version for your environment.

graphframes:(latest version)-spark(your spark version)-s_(your scala version)

I did not have to specify the jar file or copy it to the spark default jar directory when I had the right versions. Note: You need to run 'spark-shell' cmd.

%spark-shell
...
...
...
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 1.6.0
      /_/
Using Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_67)

The correct version to get for this setup from SparkPackages

For my environment I had to use the following command :

%pyspark --packages graphframes:graphframes:0.3.0-spark1.6-s_2.10