AWS EMR 5.11.0 - Apache Hive on Spark

Question

I am trying to setup Apache Hive on Spark on AWS EMR 5.11.0. Apache Spark Version - 2.2.1 Apache Hive Version - 2.3.2 Yarn logs show below error:

18/01/28 21:55:28

Answer 1

The sbt or pom.xml to be like as follows.

"org.apache.spark" %% "spark-streaming" % sparkVersion % "provided",

"org.apache.spark" %% "spark-sql" % sparkVersion % "provided",

"org.apache.spark" %% "spark-hive" % sparkVersion % "provided",

I am running DataWarehouse (Hive) on EMR and spark application stored the data into DWH.