SparkContext.addJar does not work in local mode

Question

When there is a jar file needed in a spark job, it needs to be added into spark job through 2 ways:
1. --jar path option in command.
2. SparkConte

Answer 1

Well, after some research, I found the reason. Just post here if there is someone else involved into this problem.

Method addJar() does not add jars into driver's classpath. What the method does is to find jars in driver node, distribute into worker nodes and then add into executors' classpath.
Because I submit my spark job in local mode, driver classpath (I guess) is used in the spark job, the jars added by method addJar() cannot be found.

In order to solve this problem, use --jars option to include all jars when submit the spark job or use --driver-class-path to add jars.
More details can be found here.