assembly-merge-strategy issues using sbt-assembly

Question

I am trying to convert a scala project into a deployable fat jar using sbt-assembly. When I run my assembly task in sbt I am getting the following error:

Mer         


        

Answer 1

As for the current version 0.11.2 (2014-03-25), the way to define the merge strategy is different.

This is documented here, the relevant part is:

NOTE: mergeStrategy in assembly expects a function, you can't do

mergeStrategy in assembly := MergeStrategy.first

The new way is (copied from the same source):

mergeStrategy in assembly <<= (mergeStrategy in assembly) { (old) =>
  {
    case PathList("javax", "servlet", xs @ _*)         => MergeStrategy.first
    case PathList(ps @ _*) if ps.last endsWith ".html" => MergeStrategy.first
    case "application.conf" => MergeStrategy.concat
    case "unwanted.txt"     => MergeStrategy.discard
    case x => old(x)
  }
}

This is possibly applicable to earlier versions as well, I don't know exactly when it has changed.

Answer 2

I think it should be MergeStrategy.first with a capital M, so mergeStrategy in assembly := MergeStrategy.first.

Answer 3

Quick update: mergeStrategy is deprecated. Use assemblyMergeStrategy. Apart from that, earlier responses are still solid

Answer 4

For the new sbt version (sbt-version :0.13.11), I was getting the error for slf4j; for the time being took the easy way out : Please also check the answer here Scala SBT Assembly cannot merge due to de-duplication error in StaticLoggerBinder.class where sbt-dependency-graph tool is mentioned which is pretty cool to do this manually

assemblyMergeStrategy in assembly <<= (assemblyMergeStrategy in assembly) {
  (old) => {
    case PathList("META-INF", xs @ _*) => MergeStrategy.discard
    case x => MergeStrategy.first
  }
}

Answer 5

Add following to build.sbt to add kafka as source or destination

 assemblyMergeStrategy in assembly := {
 case PathList("META-INF", xs @ _*) => MergeStrategy.discard
 //To add Kafka as source
 case "META-INF/services/org.apache.spark.sql.sources.DataSourceRegister" => 
 MergeStrategy.concat
 case x => MergeStrategy.first
 }

Answer 6

I have just setup a little sbt project that needs to rewire some mergeStrategies, and found the answer a little outdated, let me add my working code for versions (as of 4-7-2015)

• sbt 0.13.8
• scala 2.11.6

• assembly 0.13.0

  mergeStrategy in assembly := {
    case x if x.startsWith("META-INF") => MergeStrategy.discard // Bumf
    case x if x.endsWith(".html") => MergeStrategy.discard // More bumf
    case x if x.contains("slf4j-api") => MergeStrategy.last
    case x if x.contains("org/cyberneko/html") => MergeStrategy.first
    case PathList("com", "esotericsoftware", xs@_ *) => MergeStrategy.last // For Log$Logger.class
    case x =>
       val oldStrategy = (mergeStrategy in assembly).value
       oldStrategy(x)
  }