问题
Spark json4s[java.lang.NoSuchMethodError:
org.json4s.jackson.JsonMethods$.parse(Lorg/json4s/Js]
Getting above error while parsing complex json when running spark scala structured streaming application on aws emr.
回答1:
It looks like a binary compatibility error... Could you please check the dependency tree for incompatible versions of json4s artifacts?
If you will not able to upgrade them to use the same version then possible you can solve the problem by shading some of them with sbt-assembly plugin using rules like these.
In any case I would recommend to use more safe and efficient parser like jsoniter-scala (or dijon that is based on it).
来源：https://stackoverflow.com/questions/57660602/is-json4s-compatible-with-spark-2-4-0-and-emr-5-26-0