Ivy Custom Resolvers for Git or TFS

Question

I'm fairly new to Ivy, so perhaps there's a straight forward way that I'm not able to find in the documentation or what I'm looking for is not possible, but here goes. I want to be able to specify dependencies where the source code is located on local and/or remote servers which use different protocols.

Specifically, I have some project dependencies that are stored on a local network TFS server and additional project dependencies stored on a remote Git server (more precisely www.github.com). Is it possible to have Ivy download the source code and build a jar file which would then be used as the dependency? If so, how?

Answer 1

This is either impossible or would require serious hacks. Ivy is designed to download dependencies which have already been compiled. With any of the modern build management tools (Maven, Ivy, Gradle, etc.) you are always better off sticking to convention when possible. The farther you stray from the well-beaten path, the more pain you are going to cause yourself in the end.

Why do you need to download and compile the code dynamically? If you just want to have the source code available for debugging purposes, this question may help. If the reason is because you always want the absolute bleeding edge version of whatever is checked into your VCS, you would be better off setting up some sort of continuous build which produces your artifacts for you.

Answer 2

I would echo @dbyrne's answer that ivy is designed to manage binary pre-compiled dependencies. You are best served in having a local repository manager like Nexus to store your project's 3rd party dependencies.

However....

It is technically possible for ivy to download and compile dependencies, using the packager resolver. This very clever resolver is designed to take a 3rd party zip or tar archive and then use ANT to extract out the required artifact.

Below is my example which downloads the "hello world" source from the github leachim6 repository and compiles it on the fly into a jar called "hello-world.jar".

Files

Project files

|-- build.xml
|-- ivysettings.xml
|-- ivy.xml
`-- repository
    `-- leachim6
        `-- hello-world
            `-- 1.0
                `-- packager.xml

ivy.xml

<ivy-module version="2.0">
    <info organisation="com.demo" module="packager_demo"/>
    <dependencies>
        <dependency org="leachim6" name="hello-world" rev="1.0"/>
    </dependencies>
</ivy-module>

Declares a dependency against version 1.0 of the "hello-world" artifact. At this level ivy would be expected to fetch a jar from some sort of 3rd party repository.

ivysettings.xml

<ivysettings>
    <settings defaultResolver="central"/>
    <resolvers>
        <ibiblio name="central" m2compatible="true"/>
        <packager name="packager" buildRoot="${user.home}/.ivy2/packager/build" resourceCache="${user.home}/.ivy2/packager/cache" preserveBuildDirectories="false" restricted="false">
            <ivy pattern="file:///${ivy.settings.dir}/repository/[organisation]/[module]/[revision]/ivy.xml"/>
            <artifact pattern="file:///${ivy.settings.dir}/repository/[organisation]/[module]/[revision]/packager.xml"/>
        </packager>
    </resolvers>
    <modules>
        <module organisation="leachim6" name="hello-world" resolver="packager"/>
    </modules>
</ivysettings>

This is where we define the repositories (or resolvers in ivy-speak) to be used. By default ivy retrieves from Maven Central, however, we've additionally specified that the "hello-world" module should instead be retrieved using a packager resolver.

The packager attributes need some further explanation:

• resourceCache : This is where the downloaded archive is stored.
• buildRoot : This is where the downloaded archive is decompressed and acted upon.
• restricted : Normally a packager is not allowed to run ANT commands like javac and delete. Considered dangerous if the packager file was located on a remote location.

packager.xml

<packager-module version="1.0">
    <property name="name" value="${ivy.packager.module}"/>
    <property name="version" value="${ivy.packager.revision}"/>

    <resource dest="archive" url="https://github.com/leachim6/hello-world/tarball/master" sha1="7f0e2836d1e8dc6130cca68d29b6f86027b22983" type="tar.gz"/>

    <build>
        <mkdir dir="classes"/>
        <javac srcdir="archive/leachim6-hello-world-38f6567/j" includes="*.java" destdir="classes"/>
        <jar destfile="artifacts/jars/${name}.jar" basedir="classes"/>
    </build>
</packager-module>

Here is where we place the ANT script logic that creates the "hello-world.jar" artifact.

This file is used to generate an ANT script that downloads the remote artifact (using it's checksum for security) and extracts out or in our case compiles the artifact that will be returned to the ivy task.

Final notes:

• This is a contrived example. In real life, you'll discover the compile logic is normally more complicated than a simple call to javac.
• This sample will break as soon as the github project releases another version. (There were no release tags to base this sample on)