Spark uses s3a: java.lang.NoSuchMethodError

Question

I\'m doing something about the combination of spark_with_hadoop2.7 (2.4.3), hadoop (3.2.0) and Ceph luminous. When I tried to use spark to access ceph (for example, start

Answer 1

All the hadoop-* JARs need to be 100% matching on versions, else you get to see stack traces like this.

For more information please reread