how to use Cassandra Context in spark 2.0

Question

In previous Version of Spark like 1.6.1, i am using creating Cassandra Context using spark Context,

import org.apache.spark.{ Logging, SparkContext, SparkConf }
         


        

Answer 1

Short Answer: You don't. It has been deprecated and removed.

Long Answer: You don't want to. The HiveContext provides everything except for the catalogue and supports a much wider range of SQL(HQL~). In Spark 2.0 this just means you will need to manually register Cassandra tables use createOrReplaceTempView until an ExternalCatalogue is implemented.

In Sql this looks like

spark.sql("""CREATE TEMPORARY TABLE words
     |USING org.apache.spark.sql.cassandra
     |OPTIONS (
     |  table "words",
     |  keyspace "test")""".stripMargin)

In the raw DF api it looks like

spark
 .read
 .format("org.apache.spark.sql.cassandra")
 .options(Map("keyspace" -> "test", "table" -> "words"))
 .load
 .createOrReplaceTempView("words")

Both of these commands will register the table "words" for SQL queries.