Maven Shade Plugin causes duplicate jars on classpath when running integration tests

问题

I have a project which includes the S3 dependency from AWS's Java v2 SDK and builds a shaded jar. I am hitting this problem when running my integration tests from the terminal. The problem is that the interceptors are added twice because the classpath contains two jars with the S3 jar: once in my shaded jar and once from the local .m2 repository. Unfortunately I don't have any control of the code that contains this issue so I need to find a workaround until the issue is fixed.

I have replicated the problem with the following pom and test class:

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.stu</groupId>
    <artifactId>duplicate-jars-classpath</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <aws.sdk.version>2.5.62</aws.sdk.version>
        <junit.version>5.4.2</junit.version>
        <maven.failsafe.plugin.version>2.22.0</maven.failsafe.plugin.version>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>software.amazon.awssdk</groupId>
                <artifactId>bom</artifactId>
                <version>${aws.sdk.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>org.junit.jupiter</groupId>
                <artifactId>junit-jupiter-api</artifactId>
                <version>${junit.version}</version>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>org.junit.jupiter</groupId>
                <artifactId>junit-jupiter-engine</artifactId>
                <version>${junit.version}</version>
                <scope>test</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>s3</artifactId>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>2.3</version>
                <configuration>
                    <createDependencyReducedPom>false</createDependencyReducedPom>
                </configuration>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>8</source>
                    <target>8</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>${maven.failsafe.plugin.version}</version>
                <configuration>
                    <includes>
                        <include>**/*Test.java</include>
                    </includes>
                </configuration>
                <executions>
                    <execution>
                        <id>run-tests</id>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>


</project>

Test class - src/test/java/com/stu/S3DuplicateJarTest.java

package com.stu;

import java.util.Collections;

import org.junit.jupiter.api.Test;

import software.amazon.awssdk.core.interceptor.ClasspathInterceptorChainFactory;

public class S3DuplicateJarTest {

    @Test
    void check_for_duplicate_jars() throws Exception {
        System.out.println("********** AWS execution interceptors:");
        Collections.list(
                new ClasspathInterceptorChainFactory().getClass().getClassLoader()
                        .getResources("software/amazon/awssdk/services/s3/execution.interceptors")
        ).forEach(System.out::println);
    }
}

If I run the tests in my IDE then this is the output:

********** AWS execution interceptors:
jar:file:/Users/<name>/.m2/repository/software/amazon/awssdk/s3/2.5.48/s3-2.5.48.jar!/software/amazon/awssdk/services/s3/execution.interceptors

From the terminal this is the output when running mvn clean verify:

[INFO] Running com.stu.S3DuplicateJarTest
********** AWS execution interceptors:
jar:file:/Users/<name>/Development/duplicate-jars-classpath/target/duplicate-jars-classpath-1.0-SNAPSHOT.jar!/software/amazon/awssdk/services/s3/execution.interceptors
jar:file:/Users/<name>/.m2/repository/software/amazon/awssdk/s3/2.5.62/s3-2.5.62.jar!/software/amazon/awssdk/services/s3/execution.interceptors

As you can see the terminal has found two resources matching the path.

Is there anything I can do to avoid this? Is there something wrong with my configuration?

回答1:

It's due to your maven configuration, shade plugin packs all dependencies into a single jar file.

Your failsafe plugin runs tests using class path from both the project dependencies (.m2/...) and that single jar file, hence the duplicated resources.

This seems to be only happening when using failsafe in command line, though. And it's fairly easy to get around, you can simply tell failsafe to not load that dependency. (It'll be available in that single jar file anyway)

  <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-failsafe-plugin</artifactId>
      <version>${maven.failsafe.plugin.version}</version>
      <configuration>
          <includes>
              <include>**/*Test.java</include>
          </includes>
          <classpathDependencyExcludes>
              <classpathDependencyExcludes>software.amazon.awssdk:s3</classpathDependencyExcludes>
          </classpathDependencyExcludes>
      </configuration>
      <executions>
          <execution>
              <id>run-tests</id>
              <phase>integration-test</phase>
              <goals>
                  <goal>integration-test</goal>
                  <goal>verify</goal>
              </goals>
          </execution>
      </executions>
  </plugin>

来源：https://stackoverflow.com/questions/56587711/maven-shade-plugin-causes-duplicate-jars-on-classpath-when-running-integration-t