Code coverage result is not accurate to real coverage in Xcode 7

Question

I am running test cases in application with enabled code coverage data Xcode 7 Beta 2. But I am able to get only few files coverage data while my all test cases are running succ

Answer 1

I checked at Apple developers forums for the subject and after reading through various posts I guess I came across the solution.

In order for the thing to work it is necessary to:

1. Remove all your application source files from the test target
2. In your unit-test sources put @testable import <NameOfYourModule>
3. Re-build and re-run tests

I tested this out with my current project, and the results are much better.

Original solution recipe can be found at: http://natashatherobot.com/swift-2-xcode-7-unit-testing-access/

Also it seems that the functionality is a bit raw-ish, hence bugs possible, and Apple suggests submitting bug reports when things do not work as expected:

I've personally seen code coverage results for some very large projects. Right now the support works best for applications and frameworks. If that's what you're testing, then it would be best if you could file a bug report at https://bugreport.apple.com so that we can investigate your particular circumstances. For that matter, a bug report would be a good thing no matter what type of project you have. If possible, it's best to actually in the project with the report. If you can't do that, describe its setup in as much detail as possible. Pictures are good.

Original thread: https://forums.developer.apple.com/message/9733#9733

Answer 2

I think I found out what XCTest coverage ist doing and it makes some sense:

My setup:

class1 compiled into target1

class2 compiled into target1 and into target2

Test setup:

import XCTest
@testable import target1

class MyTests: XCTestCase {
    func testSomething() {
        someMethodFromClass1()
        someMethodFromClass2()
    }
}

What I find is that class1 (compiled into target1) shows test coverage and class2 (compiled into target1 and into target2) shows no test coverage.

So it appears to me that a tested method shows a coverage of 0 when there exists at least one target where this method is not tested.

And this makes a lot of sense, because testing a method in a target doesn't say anything about how it behaves in a different target.

Apple wants us to test all targets.

Update One more hint to back this theory:

go to the report navigator

and click on coverage.

If you have more than one target, you see your files grouped by target.

And if you have one file in two targets, you see your file twice.

If you have one file in both targets, the code coverage of this one file is shown for both targets. And (at least in my projects) one file has different blue lines in each target:

coverage in target 1:

coverage of same file in the same project in the same test run in target 2:

If you look at your test coverage in the source editor, apple has to decide which coverage it shows to you. I think showing the target with the lowest coverage is the best apple can do in the source editor.

simple fix for a special case:

If your only second target is your test target: don't compile into your test target and use @testable import.

For all other cases you have to test each target.

Answer 3

IT WORKS.

1. Since Apple released @testable keyword to import your project into test target, you don't have to add your files to both target anymore:

2. So just remove every file from your test target:

3. Wherever you need access to your file from your test target just import your target using: @testable import MyApp

4. Do this for every file in your project.

Then code coverage will be working fine.

Read more from Swift 2 + Xcode 7: Unit Testing Access Made Easy!!!!

If you need to know how to work with code coverage read How to use code coverage in Xcode 7?

As @Gerd Castan mentioned earlier is: "So it appears to me that a tested method shows a coverage of 0 when there exists at least one target where this method is not tested."

Solution is simple. Do not let compiler think that this file is included in more that one target, instead import your module using @testable keyword.

Answer 4

Bear in mind that there are multiple ways to cover code with tests, you may test all functions, or you may be covering all instructions in the functions, but you may not be covering all the execution paths that are possible.

Or Xcode coverage stuff may be broken, but it's hard to tell if you don't give details on what kind of coverage are you expecting it to check.

Answer 5

This happens because .swift file of your project selected for both targets by default.

Manually select and remove test target for files works for me.