Is there a way to “fail fast” for junit with the maven surefire plugin?

Question

I\'m currently working on a java project using maven. We use the maven surefire plugin to run our junit suite as part of the build process.

Our test suite is rapidl

Answer 1

As far as I know, no, and this really requires the resolution of SUREFIRE-580. If you want to make this happen faster, you should at least vote for the issue and, optionally, submit a patch ;)

Answer 2

A few ways to speed it up if not exactly what you need:

• If it's a multi module build, add --fail-fast to the command line to drop out after the first module.

• Look into failsafe to move long running integration tests onto a different phase of the lifecycle.

• Look into a profile based solution for fast and slow tests - Is there a way to tell surefire to skip tests in a certain package?.

Answer 3

As of September 6th 2015 it appears there is, -Dsurefire.skipAfterFailureCount=1.

As of October 19th 2015 version 2.19 has been released.

Answer 4

There may be a suitable workaround, but it depends on your requirements and you need to use a CI server which can handle jvm process return codes.

The basic idea is to stop Maven's JVM process altogether and let the OS know that the process has stopped unexpectedly. Then, a continuous integration server like Jenkins/Hudson should be able to check for a non-zero exit code and let you know that a test has failed.

The first step is to make surefire exit the JVM at the first test failure. You can do that with JUnit 4.7 or higher by using a custom RunListener (put it in src/test/java):

package org.example
import org.junit.runner.notification.Failure;
import org.junit.runner.notification.RunListener;
public class FailFastListener extends RunListener {
        public void testFailure(Failure failure) throws Exception {
                System.err.println("FAILURE: " + failure);
                System.exit(-1);
        }
}

Then, you need to configure that class so surefire will register it with the JUnit 4 Runner. Edit your pom.xml and add the listener configuration property to the maven-surefire-plugin. You will also need to configure surefire to not fork a new JVM process for executing the tests. Otherwise, it will just go on with the next test cases.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.10</version>
    <configuration>
        <forkMode>never</forkMode>
        <properties>
            <property>
                <name>listener</name>
                <value>org.example.FailFastListener</value>
            </property>
        </properties>
    </configuration>
</plugin>

If this does not help, I'd try to fork the maven surefire junit provider plugin.

Btw, unit tests, by definition, should run faster than 0.1 seconds. If your build really takes so long due to the unit tests, you will have to make them run faster in the future.

Answer 5

You can run maven with --fail-fast option:

The -ff option is very useful for developers running interactive builds who want to have rapid feedback during the development cycle.

an example can be:

mvn clean test -ff

http://books.sonatype.com/mvnref-book/reference/running-sect-options.html

Answer 6

This isn't a direct answer to the question, but it can also be useful to feed the output of maven through grep, to strip out most stuff and help you to see where the test failures are.

Like this:

mvn test | grep -w 'Running\|Tests'

Which produces output (for my code) like this:

Running scot.mygov.pp.test.dashboard.DashboardJsonSerDesCRUDTest
Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec
Running scot.mygov.pp.test.dashboard.DashboardBaselineCRUDTest
Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.026 sec
Running scot.mygov.pp.test.dashboard.DashboardDatabaseValidationTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.264 sec
Running scot.mygov.pp.test.dashboard.DashboardServiceWebServiceIsolationCRUDTest
Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.032 sec

Much easier to see where the first error of failure is.