How to use JMH with gradle?

Question

I want to use JMH, an OpenJDK microbenchmark tool, with gradle. However, Im getting the NPE on compilation. On the other hand, JMH works when using from maven.<

Answer 1

Just finished my "masterpiece". No uber-jars, no plugins, code base separated from main & test, benchmarks compilation hooked to main, but does not run automatically in the mainstream lifecycle. Simple, explicit, and hackable, vanilla gradle.

I run it directly from IntelliJ, to run on a box you probably will need the uber-jar back :-)

Before doing it I have spent a fair amount of time trying to get that plugin work, but it's way too clunky for my taste.

Step-by-step breakdown below.

Define a new sourceSet called jmh with classpath hooked to that of the main sourceSet

sourceSets {
    jmh {
        java.srcDirs = ['src/jmh/java']
        scala.srcDirs = ['src/jmh/scala']
        resources.srcDirs = ['src/jmh/resources']
        compileClasspath += sourceSets.main.runtimeClasspath
    }
}

Define dependencies for it (at minimum JMH and its annotation processor).

dependencies {
    ...
    jmhImplementation 'org.openjdk.jmh:jmh-core:1.21'
    jmhImplementation 'org.openjdk.jmh:jmh-generator-annprocess:1.21'
}

Define a task jmh of type JavaExec

task jmh(type: JavaExec, dependsOn: jmhClasses) {
    main = 'org.openjdk.jmh.Main'
    classpath = sourceSets.jmh.compileClasspath + sourceSets.jmh.runtimeClasspath
}

Hook jmhClasses task to run after classes to make sure benchmarks are compiled with the rest of the code

classes.finalizedBy(jmhClasses)

Answer 2

I made a very small example project to clone and modify as you like. It's a full working example:
https://gitlab.com/barfuin/jmh-gradle-example

It requires no shadow Jars and no plugins. The project also includes some extra Gradle tasks for printing the classpath, the JMH options, etc., stuff that may help to understand what's going on. It does not teach microbenchmarking though.

Answer 3

Currently you can just use dedicated plugin jmh-gradle-plugin

It requires minimal configuration and allows you to run JMH benchmarks as well as build benchmarks artifact

Answer 4

My bad, I was trying to benchmark a method that has an argument - of course JMH will not know what to pass :) Once when I created a void method with no arguments, everything worked.

My build.gradle:

defaultTasks 'build'

apply plugin: 'java'
apply plugin: 'shadow'

buildscript {
    repositories {
        mavenCentral()
        maven {
            name 'Shadow'
            url 'http://dl.bintray.com/content/johnrengelman/gradle-plugins'
        }
    }
    dependencies {
        classpath 'org.gradle.plugins:shadow:0.7.4'
    }
}

jar {
    manifest {
        attributes 'Main-Class': 'org.openjdk.jmh.Main'
    }
}

repositories {
    mavenCentral()
}


build.doLast {
    tasks.shadow.execute()
}

shadow {
    outputFile = new File('build/libs/microbenchmarks.jar')
}

ext {
    lib = [
        ... other dependencies...
        jmh:            'org.openjdk.jmh:jmh-core:0.2'
    ]
}

dependencies {
    compile lib... other dependencies...
    compile lib.jmh
}

sourceCompatibility = 1.7

Build tests and jar:

gw clean build

and then run them with:

java -jar build/libs/microbenchmarks.jar ".*" -wi 2 -i 10 -f 2 -t 16

UPDATE

From recent versions of JMH, you would also need to add dependency to:

org.openjdk.jmh:jmh-generator-annprocess:0.5.4

and you can use shadow 0.8.

Answer 5

If you're an IntelliJ user, perhaps the easiest way to make it work, without all that workarounds, is to use the IDE plugin:

https://github.com/artyushov/idea-jmh-plugin

• Add the dependencies
• Create your benchmark
• be happy :)