One of the practice many companies follow is to repeat unstable test until is passes x times (in a row or in total). If it is executed n times and fail to pas
U can try this extension for junit 5.
io.github.artsok
rerunner-jupiter
LATEST
Examples:
/**
* Repeated three times if test failed.
* By default Exception.class will be handled in test
*/
@RepeatedIfExceptionsTest(repeats = 3)
void reRunTest() throws IOException {
throw new IOException("Error in Test");
}
/**
* Repeated two times if test failed. Set IOException.class that will be handled in test
* @throws IOException - error occurred
*/
@RepeatedIfExceptionsTest(repeats = 2, exceptions = IOException.class)
void reRunTest2() throws IOException {
throw new IOException("Exception in I/O operation");
}
/**
* Repeated ten times if test failed. Set IOException.class that will be handled in test
* Set formatter for test. Like behavior as at {@link org.junit.jupiter.api.RepeatedTest}
* @throws IOException - error occurred
*/
@RepeatedIfExceptionsTest(repeats = 10, exceptions = IOException.class,
name = "Rerun failed test. Attempt {currentRepetition} of {totalRepetitions}")
void reRunTest3() throws IOException {
throw new IOException("Exception in I/O operation");
}
/**
* Repeated 100 times with minimum success four times, then disabled all remaining repeats.
* See image below how it works. Default exception is Exception.class
*/
@DisplayName("Test Case Name")
@RepeatedIfExceptionsTest(repeats = 100, minSuccess = 4)
void reRunTest4() {
if(random.nextInt() % 2 == 0) {
throw new RuntimeException("Error in Test");
}
}
View at IDEA:
With minimum success four times then disables all other:
You can also mix @RepeatedIfExceptionsTest with @DisplayName
source -> github