Merging Integration and Unit test reports with JaCoCo

Question

I am generating failsafe and surefire reports with maven with the JaCoCo plugin, but I can only manage to get them in separate reports. I would like to have a overall covera

Answer 1

Maybe more nice way is to achieve is use same jacoco file, but let it another tests, which works for us - unit and it tests.

            <plugin>
                <groupId>org.jacoco</groupId>
                <artifactId>jacoco-maven-plugin</artifactId>
                <version>0.8.1</version>
                <executions>
                    <execution>
                        <id>default-prepare-agent</id>
                        <goals>
                            <goal>prepare-agent</goal>
                        </goals>
                        <configuration>
                            <skip>${maven.surefire.skipTests}</skip>
                            <propertyName>maven.surefire.argLine</propertyName>
                            <!-- using the same dest file for both UT and IT -->
                            <destFile>${sonar.jacoco.reportPath}</destFile>
                        </configuration>
                    </execution>
                    <execution>
                        <id>default-prepare-agent-integration</id>
                        <goals>
                            <goal>prepare-agent-integration</goal>
                        </goals>
                        <configuration>
                            <skip>${maven.failsafe.skipTests}</skip>
                            <propertyName>maven.failsafe.argLine</propertyName>
                            <!-- append to the UT dest file -->
                            <destFile>${sonar.jacoco.reportPath}</destFile>
                            <append>true</append>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <skipTests>${maven.surefire.skipTests}</skipTests>
                <failIfNoTests>${maven.surefire.failIfNoTests}</failIfNoTests>
                <!-- allow argLine to be modified by other plugins, e.g. jacoco -->
                <argLine>@{maven.surefire.argLine}</argLine>
            </configuration>
        </plugin>
        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <configuration>
                <skipTests>${maven.failsafe.skipTests}</skipTests>
                <failIfNoTests>${maven.failsafe.failIfNoTests}</failIfNoTests>
                <!-- allow argLine to be modified by other plugins, e.g. jacoco -->
                <argLine>@{maven.failsafe.argLine}</argLine>
            </configuration>
        </plugin>

Answer 2

You should take a look into the documentation of JaCoCo Maven plugin which contains a merge goal.

Answer 3

Building on top of the answer of Chad, here is my configuration. It is configured as a separate profile named jacoco so I can turn it on and off easily. It uses only the default configuration. It can create a separate code coverage report for unit tests, separate code coverage report for integration tests and also a combined code coverage report.

    <profile>
        <id>jacoco</id>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.jacoco</groupId>
                    <artifactId>jacoco-maven-plugin</artifactId>
                    <executions>
                        <execution>
                            <id>pre-unit-test</id>
                            <goals>
                                <goal>prepare-agent</goal>
                            </goals>
                        </execution>
                        <execution>
                            <id>post-unit-test</id>
                            <phase>test</phase>
                            <goals>
                                <goal>report</goal>
                            </goals>
                        </execution>
                        <execution>
                            <id>pre-integration-test</id>
                            <goals>
                                <goal>prepare-agent-integration</goal>
                            </goals>
                        </execution>
                        <execution>
                            <id>post-integration-test</id>
                            <goals>
                                <goal>report-integration</goal>
                            </goals>
                        </execution>
                        <execution>
                            <id>merge-results</id>
                            <phase>verify</phase>
                            <goals>
                                <goal>merge</goal>
                            </goals>
                            <configuration>
                                <fileSets>
                                    <fileSet>
                                        <directory>${project.build.directory}</directory>
                                        <includes>
                                            <include>*.exec</include>
                                        </includes>
                                        <excludes>
                                            <exclude>aggregate.exec</exclude>
                                        </excludes>
                                    </fileSet>
                                </fileSets>
                                <destFile>${project.build.directory}/aggregate.exec</destFile>
                            </configuration>
                        </execution>
                        <execution>
                            <id>post-merge-report</id>
                            <phase>verify</phase>
                            <goals>
                                <goal>report</goal>
                            </goals>
                            <configuration>
                                <dataFile>${project.build.directory}/aggregate.exec</dataFile>
                                <outputDirectory>${project.reporting.outputDirectory}/jacoco-aggregate</outputDirectory>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>

Answer 4

I recently implemented this: after some headaches and a lot of testing, I have a configuration that works beautifully.

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>${jacoco.version}</version>
    <executions>
        <execution>
            <id>pre-unit-test</id>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
            <configuration>
                <destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
                <propertyName>surefireArgLine</propertyName>
            </configuration>
        </execution>
        <execution>
            <id>pre-integration-test</id>
            <goals>
                <goal>prepare-agent-integration</goal>
            </goals>
            <configuration>
                <destFile>${project.build.directory}/coverage-reports/jacoco-it.exec</destFile>
                <propertyName>testArgLine</propertyName>
            </configuration>
        </execution>
        <execution>
            <id>post-integration-test</id>
            <phase>post-integration-test</phase>
            <goals>
                <goal>report</goal>
            </goals>
            <configuration>
                <dataFile>${project.build.directory}/coverage-reports/jacoco-it.exec</dataFile>
                <outputDirectory>${project.reporting.outputDirectory}/jacoco-it</outputDirectory>
            </configuration>
        </execution>
        <execution>
            <id>post-unit-test</id>
            <phase>prepare-package</phase>
            <goals>
                <goal>report</goal>
            </goals>
            <configuration>
                <dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
                <outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
            </configuration>
        </execution>
        <execution>
            <id>merge-results</id>
            <phase>verify</phase>
            <goals>
                <goal>merge</goal>
            </goals>
            <configuration>
                <fileSets>
                    <fileSet>
                        <directory>${project.build.directory}/coverage-reports</directory>
                        <includes>
                            <include>*.exec</include>
                        </includes>
                    </fileSet>
                </fileSets>
                <destFile>${project.build.directory}/coverage-reports/aggregate.exec</destFile>
            </configuration>
        </execution>
        <execution>
            <id>post-merge-report</id>
            <phase>verify</phase>
            <goals>
                <goal>report</goal>
            </goals>
            <configuration>
                <dataFile>${project.build.directory}/coverage-reports/aggregate.exec</dataFile>
                <outputDirectory>${project.reporting.outputDirectory}/jacoco-aggregate</outputDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.18.1</version>
    <configuration>
        <argLine>${surefireArgLine}</argLine>
        <skipTests>${skip.unit.tests}</skipTests>
        <includes>
            <include>**/*UT.java</include>
            <include>**/*MT.java</include>
            <include>**/*Test.java</include>
        </includes>
        <skipTests>${skipUTMTs}</skipTests>
    </configuration>
</plugin>
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.12.4</version>
    <configuration>
        <skipTests>${skipTests}</skipTests>
        <skipITs>${skipITs}</skipITs>
        <argLine>${testArgLine}</argLine>
        <excludes>
            <exclude>**/*UT*.java</exclude>
        </excludes>
    </configuration>
    <executions>
        <execution>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
</plugin>

As you can see, there are 6 distinct Jacoco executions to run the tests, merge the files and create an aggregate report. On top of the Jacoco config, you also need to configure Surefire and Failsafe to take an argument from Jacoco (Surefire runs the unit tests and Failsafe runs the integration tests).

All of the configuration that I used should be there, what you do with it is your design architecture to make it fit within your desired requirements. Personally, I recommend a look into what I exclude and include within surefire and failsafe if you are having issues with files not being read.

Answer 5

Not the answer you're looking for, but still...

In most cases, you should not merge coverage from unit and integration tests.

The value of unit tests is that they improve the design of your application and ensure that corner cases of your code is working correctly. You should try to have a high branch coverage of your unit tests.

The value of your integration tests is that they ensure that the main use cases of your application are working correctly and that the whole stack is integrated correctly. You should try to have a high functional coverage for your integration tests. (But it is fairly hard to measure functional coverage with a tool).

If you need integration tests to improve your branch coverage, that's a strong indication that you should review the design of your code. And if you already have a high branch coverage without integration tests, adding them should not modify significantly your metrics.

Feel free to down vote this answer as it is a bit of topic and fairly opinionated...