Spark SQL fails with java.lang.NoClassDefFoundError: org/codehaus/commons/compiler/UncheckedCompileException

Question

Running a Spark SQL (v2.1.0_2.11) program in Java immediately fails with the following exception, as soon as the first action is called on a dataframe:

java.lang         


        

Answer 1

in our migration from CDH Parcel 2.2.0.cloudera1 to 2.3.0.cloudera4 we have simply overwritten the maven property :

3.0.8

In addition, we have defined the proper version of the hive dependency in the dependency management part:

1.1.0-cdh5.13.3

    
         org.apache.hive
         hive-jdbc
         ${hive.version}
         runtime
         
             
                 org.eclipse.jetty.aggregate
                 *
             
             
                 slf4j-log4j12
                 org.slf4j
             
             
                 parquet-hadoop-bundle
                 com.twitter
             
         
     

The exclusions were necessary for the previous version, they might not be necessary anymore