Maven Surefire: Unable to fork parallel test execution

Question

using Maven surefire, I'm unable to fork parallel test execution. That is, each of my test cases hs to run in a serapate JVM, hence the forking. In addition, I want my test cases to run in parallel. the first part is working without problem: I'm able to run each test case in its own JVM. the second part, however is still a challene for me. I haven't managed to get the paralle execution of test cases working. Here is how my plugin declaration look like:

    <plugin>
          <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>2.5</version>
      <configuration>
           <parallel>methods</parallel>
           <forkMode>always</forkMode>
                <argLine>-Xms512m -Xmx512m</argLine>
       </configuration>
</plugin>

I've tried both methods and classes but haven't see any parallelization. My JUnit version is 4.7 as shown by the depency declaration:

    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.7</version>
        <scope>compile</scope>
    </dependency>            

Any help would be much appricated.

Gregoire.

Answer 1

I think that you are supposed to use the threadCount parameter when using the parallel mode:

  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.6</version>
    <configuration>
      <forkMode>always</forkMode>
      <argLine>-Xms512m -Xmx512m</argLine>
      <parallel>methods</parallel>
      <threadCount>4</threadCount>
    </configuration>
  </plugin>

Answer 2

I had the same problem, because i was using surefire version 2.7, after upgrade to 2.12 it worked with the following configuration:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.12</version>
  <configuration>
    <parallel>classes</parallel>
    <forkMode>perthread</forkMode>
    <threadCount>4</threadCount>
  </configuration>
</plugin>

It spawned 4 threads, each running it's own jvm.

Answer 3

Make sure you get a log message something like this

[INFO] Concurrency config is {perCoreThreadCount=false, threadCount=1, parallel=classes, configurableParallelComputerPresent=true}

Just before this heading:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------

This message indicates that the parallel surefire junit provider is active.

If this is not present surefire may be picking up a different version of junit than you think. Anything below 4.7 will not work. Run mvn dependency:tree to check which version(s) are present.

You should also upgrade to surefire 2.6 since a number of minor bugs related to parallel running have been fixed. For the same reason you should use the latest junit.

Answer 4

Surefire's parallel mode is extremely buggy. For example, see http://jira.codehaus.org/browse/SUREFIRE-747 and http://jira.codehaus.org/browse/SUREFIRE-730

I haven't managed to get a single test running in parallel to date (not to mention forking).

Answer 5

Try changing your forkMode from always to "never". It does not state this in their documentation, but you can not have fork plus parallel at this time (we found this after digging through the surefire code.)

Just so you know, you will probably run into tests that are not thread safe due to many test/supporting libraries (easymock, powermock, etc) invalidating the ability to parallel your tests.

Answer 6

Are you sure it's not working? You may not gain much speedup if your tests do not contain many test methods. With forkMode=always the best you can do is run all test methods within a class, in parallel.

Answer 7

Isn't parallel setting a TestNG only attribute? according to this: http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html#parallel

Answer 8

The surefire 2.16 fixed the parallel execution regarding the JUnit tests.