showing coverage of functional tests without blind spots

Question

I have a production golang code and functional tests for it written not in golang. Functional tests run compiled binary. Very simplified version of my production code is her

Answer 1

In my pkglint project I have declared a package-visible variable:

var exit = os.Exit

In the code that sets up the test, I overwrite it with a test-specific function, and when tearing down a test, I reset it back to os.Exit.

This is a simple and pragmatic solution that works well for me, for at least a year of extensive testing. I get 100% branch coverage because there is no branch involved at all.

Answer 2

As per our conversation in the comments our coverage profile will never include that line of code because it will never be executed.

With out seeing your full code it is hard to come up with a proper solutions however there a few things you can do to increase the coverage with out sacrificing too much.

func Main and TestMain

Standard practice for GOLANG is to avoid testing the main application entry point so most professionals extract as much functionality into other classes so they can be easily tested.

GOLANG testing framework allows you to test your application with out the main function but in it place you can use the TestMain func which can be used to test where the code needs to be run on the main thread. Below is a small exert from GOLANG Testing.

It is sometimes necessary for a test program to do extra setup or teardown before or after testing. It is also sometimes necessary for a test to control which code runs on the main thread. To support these and other cases, if a test file contains a function: func TestMain(m *testing.M)

Check GOLANG Testing for more information.

Working example

Below is an example (with 93.3% coverage that we will make it 100%) that tests all the functionality of your code. I made a few changes to your design because it did not lend itself very well for testing but the functionality still the same.

package main

dofunc.go

import (
    "fmt"
    "math/rand"
    "time"
)

var seed int64 = time.Now().UTC().UnixNano()

func doFunc() int {
    rand.Seed(seed)
    var code int
    for {
        i := rand.Int()
        fmt.Println(i)
        if i%3 == 0 {
            code = 0
            break
        }
        if i%2 == 0 {
            fmt.Println("status 1")
            code = 1
            break
        }
        time.Sleep(time.Second)
    }
    return code
}

dofunc_test.go

package main

import (
    "testing"
    "flag"
    "os"
)

var exitCode int

func TestMain(m *testing.M) {
    flag.Parse()
    code := m.Run()
    os.Exit(code)
}

func TestDoFuncErrorCodeZero(t *testing.T) {
    seed = 2

    if code:= doFunc(); code != 0 {
        t.Fail()
    }
}

func TestDoFuncErrorCodeOne(t *testing.T) {
    seed = 3

    if code:= doFunc(); code != 1 {
        t.Fail()
    }
}

main.go

package main

import "os"

func main() {
    os.Exit(doFunc());
}

Running the tests

If we build our application with the cover profile.

$ go test -c -coverpkg=. -o example

And run it.

$ ./example -test.coverprofile=/tmp/profile

Running the tests

1543039099823358511
2444694468985893231
6640668014774057861
6019456696934794384
status 1
PASS
coverage: 93.3% of statements in .

So we see that we got 93% coverage we know that is because we don't have any test coverage for main to fix this we could write some tests for it (not a very good idea) since the code has os.Exit or we can refactor it so it is super simple with very little functionality we can exclude it from our tests.

To exclude the main.go file from the coverage reports we can use build tags by placing tag comment at the first line of the main.go file.

//+build !test

For more information about build flags check this link: http://dave.cheney.net/2013/10/12/how-to-use-conditional-compilation-with-the-go-build-tool

This will tell GOLANG that the file should be included in the build process where the tag build is present butNOT where the tag test is present.

See full code.

//+build !test

package main

import "os"

func main() {
    os.Exit(doFunc());
}

We need need to build the the coverage application slightly different.

$ go test -c -coverpkg=. -o example -tags test

Running it would be the same.

$ ./example -test.coverprofile=/tmp/profile

We get the report below.

1543039099823358511
2444694468985893231
6640668014774057861
6019456696934794384
status 1
PASS
coverage: 100.0% of statements in .

We can now build the coverage html out.

$ go tool cover -html /tmp/profile -o /tmp/profile.html