How to get 100% Code Coverage with PHPUnit

Question

I am writing a Zend Framework application and unit testing it with PHPUnit. In general, things are going swimmingly however I have a small, but annoying issue with PHPUnit

Answer 1

The code you have only comments for is what matters. The closing brace of a block will be shown as executable in the code coverage report if it's possible to fall through to the end. Look for a branch that you aren't actually testing.

if ($request->isPost()) {
    if ($x < 5) {
        return '<';
    }
    elseif ($x > 5) {
        return '>';
    }
    // Do you have a test for $x == 5?
}

As for the 100% code coverage goal, I agree wholeheartedly with Bill. For some framework classes that I write I will strive to make 100%, but I know that doesn't mean I've truly tested every possibility. Often, when I find myself working overly hard to achieve 100% coverage, it's probably OCD kicking in. :)

Just . . . one . . . more . . . test . . .

Answer 2

I was the project lead on the Zend Framework a few years ago, through the release of ZF 1.0. I worked pretty hard on raising the coverage of testing for all components, and we had a policy that a component must have a certain minimum code coverage to be adopted into ZF from the incubator.

However, you're right, trying to get 100% code coverage from tests for all your classes isn't really practical. Some of the classes in ZF have 100% coverage, but for these, one or more of the following was true:

• The class was trivially simple.
• The tests took extraordinary work to write. E.g. complex setup code to create conditions to exercise all the obscure corner cases. Look at the unit tests for Zend_Db that I wrote! Though it's beneficial to force yourself to test these corner cases, because I guarantee that it'll lead you to code that you need to fix.
• The class had to be refactored to be more "testable". This is often a good thing anyway, because you end up with better OO code, with less coupling, fewer static elements, etc. See the classes and tests for Zend_Log.

But we also realized that 100% code coverage is sometimes an artificial goal. A test suite that achieves less tan 100% coverage may nevertheless be adequate. And a test suite that does get to 100% coverage doesn't necessarily assure quality.

It would be very easy to get 100% code coverage for the following function. But did we test for division by zero?

function div($numerator, $denominator) {
    return $numerator / $denominator;
}

So you should use code coverage as one metric of testing, but not the ultimate goal.

Answer 3

If that is all there is to your test than i would assume your tests looks like Matthew described:

class UserControllerTest extends Zend_Test_PHPUnit_ControllerTestCase {
    // [...]
    public function testSomething()
    {
        $this->request
             ->setMethod('POST')
             ->setPost(array(
                 'username' => 'foobar',
                 'password' => 'foobar'
             ));
        $this->editAction();
        // assertThatTheRightThingsHappend
    }
}

and in that case i don't see any reason why it shouldn't be easy to get 100% Code Coverage.

But yes: It is pretty hard to test Zend Framework Controllers and at some point you either have to try really hard to get all your application logic out of your controllers or just live with it.

The same thing doesn't apply for your models though. Those should be really easy to test, even in a ZF Application.

The purpose that code coverage serves is that it tells you what parts of your code base are not even getting executed. It doesn't tell you what is really tested and can only serve as a "minimum" to get an idea about the quality of your test suite (if you don't use @covers even that might lie to you).

On short: If you have big controllers and an not so easy to change architecture just setting with so and so tested controllers but don't apply the same logic to your models. Nothing in ZF prevents you to properly test those