How to work efficiently with SBT, Spark and “provided” dependencies?

Question

I\'m building an Apache Spark application in Scala and I\'m using SBT to build it. Here is the thing:

1. when I\'m developing under IntelliJ IDEA, I want Spark depend

Answer 1

For running the spark jobs, the general solution of "provided" dependencies work: https://stackoverflow.com/a/21803413/1091436

You can then run the app from either sbt, or Intellij IDEA, or anything else.

It basically boils down to this:

run in Compile := Defaults.runTask(fullClasspath in Compile, mainClass in (Compile, run), runner in (Compile, run)).evaluated,
runMain in Compile := Defaults.runMainTask(fullClasspath in Compile, runner in(Compile, run)).evaluated