Can I avoid running junit tests twice in eclipse when using a TestSuite?

Question

I need to do some per-suite initialisation (starting a web-server). It is working fine except that when I run all tests in my project in eclipse my tests run twice. My test

Answer 1

I realize that this has been asked over 5 years ago, but as quite a few folks up-voted the question I thought I'd still chime in with a solution. Skip right to the end if you just want the solution; read the whole text if you also want to understand it ;-)

First of all, it is indeed possible to ensure that a particular JUnit test class gets only run inside a test suite. Also, it is irrelevant whether you want to run that test suite inside Eclipse (as asked here) or any other tool or environment; this is really a pure JUnit issue for the most part.

Before I sketch out the solution, it might be a good idea to revisit what the exact problem is here. All JUnit tests need to be visible and instantiable to be picked up by the JUnit framework and its various runners. This also applies to test suites and the individual tests that are part of a test suite. As a consequence, if JUnit picks up the test suite it will also pick up the individual tests, and all tests in the suite will be executed twice, once individually and once as part of the suite.

So, the trick, if you will, is to prevent JUnit from picking up the individual tests while still being able to instantiate and execute them as part of the suite.

One thing that comes to mind is to make the test classes static inner classes nested inside the test suite. However, the nested classes still need to be public (otherwise they can't be run in the suite either), and if they are public classes they will also be picked up individually, despite being nested inside the suite's public class. JUnit will not try to run test classes that are not considered visible, though. So, nesting the test classes inside a non-public class would presumably be sufficient to hide them, but we can't make the suite class non-public because then JUnit would not execute it. What we can do, however, is to nest the individual tests inside another non-public class that's nested inside the test suite, which leads us to the solution of this conundrum:

import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;

@RunWith(Suite.class)
@SuiteClasses({AllTests.InSuiteOnly.Test1.class, AllTests.InSuiteOnly.Test2.class})
public class AllTests
{
    static class InSuiteOnly
    {
        public static class Test1
        {
            @Test
            public void test1()
            {
                //...
            }
        }

        public static class Test2
        {
            @Test
            public void test2()
            {
                //...
            }
        }
    }
}

A lot of folks will probably object to all tests needing to be inside a single source file now. What if I want to maintain separate JUnit test classes that don't get executed by themselves but still get executed inside the test suite? A simple solution is to make the individual test classes abstract (public/non-public doesn't matter) so that JUnit won't execute them, and inside the test suite we simply use concrete subclasses of the original abstract test classes:

import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;

@RunWith(Suite.class)
@SuiteClasses({AllTests.InSuiteOnly.SuiteTest1.class, AllTests.InSuiteOnly.SuiteTest2.class})
public class AllTests
{
    static class InSuiteOnly
    {
        public static class SuiteTest1 extends Test1 {}
        public static class SuiteTest2 extends Test2 {}
    }
}

abstract class Test1
{
    @Test
    public void test1()
    {
        //...
    }
}

abstract class Test2
{
    @Test
    public void test2()
    {
        //...
    }
}

This scheme works with Maven, Eclipse, and all other environments that either directly leverage JUnit's runners or implement their own runners that closely follow JUnit's original behavior and semantics.

Answer 2

do you need the suite in the first place ? depending on when you click for run all (class, package, or src/test/java), all underlying tests will be executed. So what's the point of having a suite ?

Answer 3

I have an idea for you. Actually you do not want to run these test case as stand-alone test cases. You can do the following.

Mark the test cases with annotation @RunWith(DoNothingRunner.class)

Implment DoNothingRunner as following:

public class DoNothingRunner extends Runner {
    public Description getDescription() {
              return "do nothing";
        }
    public void run(RunNotifier notifier) {
            // indeed do nothing
        }
}

I have not tried this personally but I hope this will work.

Answer 4

There is a solution, it's a bit tricky, but it may easily resolve your problem: create one suite class, and include all your suite classes in it. Then you can use this suite class to run all your tests.

@RunWith(Suite.class)
@Suite.SuiteClasses({
    AXXSuite.class,
    BXXSuite.class,
    CXXSuite.class
})
public class AllSuites {    

}

Answer 5

No, the test class will always be started directly and then through the "link" in the suite. This is as expected.

One workaround might to set in the run configuration to only run tests from the package which contains your suites. Open the run configuration and select Run all tests in the selected project, package or source folder then click Search... and select the package.