Can I have multiple spark versions installed in CDH?

Question

I\'m using cdh5.1.0, which already has default spark installed. However, I want to use Spark 1.3. Can I also install this version to cdh5.1.0? How is it possible to set thes

Answer 1

Yes, you can run any Apache Spark version you like. Just make sure it's built for the version of YARN you have (2.3 for CDH 5.1.0). You can then run your application as a YARN application with spark-submit. (See http://spark.apache.org/docs/latest/running-on-yarn.html.) It will be monitored like any other YARN application.

Spark doesn't need to be "installed" at all, you just run your application.