How to cache maven dependencies in Docker

Question

I\'m working on a project with ~200MB dependencies and i\'d like to avoid useless uploads due to my limited bandwidth.

When I push my Dockerfile (i\'ll attach it in a m

Answer 1

If I understand correctly what you'd like to achieve, the problem is to avoid creating a fat jar with all Maven dependencies at each Docker build (to alleviate the size of the Docker layers to be pushed after a rebuild).

If yes, you may be interested in the Spring Boot Thin Launcher, which is also applicable for non-Spring-Boot projects. Some comprehensive documentation is available in the README.md of the corresponding GitHub repo: https://github.com/dsyer/spring-boot-thin-launcher#readme

To sum up, it should suffice to add the following plugin declaration in your pom.xml:

<build>
    <plugins>
        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
            <!--<version>${spring-boot.version}</version>-->
            <dependencies>
                <dependency>
                    <groupId>org.springframework.boot.experimental</groupId>
                    <artifactId>spring-boot-thin-layout</artifactId>
                    <version>1.0.19.RELEASE</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>

Ideally, this solution should be combined with a standard Dockerfile setup to benefit from Docker's cache (see below for a typical example).

Leverage Docker's cache mechanism for a Java/Maven project

The archetype of a Dockerfile that avoids re-downloading all Maven dependencies at each build if only source code files (src/*) have been touched is given in the following reference:
https://whitfin.io/speeding-up-maven-docker-builds/

To be more precise, the proposed Dockerfile is as follows:

# our base build image
FROM maven:3.5-jdk-8 as maven

WORKDIR /app

# copy the Project Object Model file
COPY ./pom.xml ./pom.xml

# fetch all dependencies
RUN mvn dependency:go-offline -B

# copy your other files
COPY ./src ./src

# build for release
# NOTE: my-project-* should be replaced with the proper prefix
RUN mvn package && cp target/my-project-*.jar app.jar


# smaller, final base image
FROM openjdk:8u171-jre-alpine
# OPTIONAL: copy dependencies so the thin jar won't need to re-download them
# COPY --from=maven /root/.m2 /root/.m2

# set deployment directory
WORKDIR /app

# copy over the built artifact from the maven image
COPY --from=maven /app/app.jar ./app.jar

# set the startup command to run your binary
CMD ["java", "-jar", "/app/app.jar"]

Note that it relies on the so-called multi-stage build feature of Docker (presence of two FROM directives), implying the final image will be much smaller than the maven base image itself.
(If you are not interested in that feature during the development phase, you can remove the lines FROM openjdk:8u171-jre-alpine and COPY --from=maven /app/app.jar ./app.jar.)

In this approach, the Maven dependencies are fetched with RUN mvn dependency:go-offline -B before the line COPY ./src ./src (to benefit from Docker's cache).

Note however that the dependency:go-offline standard goal is not "perfect" as a few dynamic dependencies/plugins may still trigger some re-downloading at the mvn package step. If this is an issue for you (e.g. if at some point you'd really want to work offline), you could take at look at that other SO answer that suggests using a dedicated plugin that provides the de.qaware.maven:go-offline-maven-plugin:resolve-dependencies goal.

Answer 2

In general Dockerfile container build, works in layers and each time you build these layers are available in catch and is used if there are no changes. Ideally it should have worked same way.

Maven generally looks for dependencies by default in .m2 folder located in Home dir of User in Ubuntu /home/username/

If dependent jars are not available then it downloads those jars to .m2 and uses it.

Now you can zip and copy this .m2 folder after 1 successful build and move it inside Docker Container User's Home directory.

Do this before you run build command

Note: You might need to replace existing .m2 folder in docker

So your Docker file would be something like this

FROM maven:3.6.0-jdk-8-slim

WORKDIR /app

COPY .m2.zip /home/testuser/

ADD pom.xml /app

RUN mvn verify clean --fail-never

COPY ./src /app/src

RUN mvn package
...

Answer 3

The documentation of the official Maven Docker images also points out different ways to achieve better caching of dependencies.

Basically, they recommend to either mount the local maven repository as a volume and use it across Docker images or use a special local repository (/usr/share/maven/ref/) the contents of which will be copied on container startup.