How do you tell that your unit tests are correct?

Question

I\'ve only done minor unit testing at various points in my career. Whenever I start diving into it again, it always troubles me how to prove that my tests are correct. How

Answer 1

I guess writing the test first (before writing the code) is a pretty good way of being sure your test is valid.

Or you could write tests for your unit tests... :P

Answer 2

Edit: I also realize that you could write small, granular unit tests that would be easy to understand. However, if you assume that small, granular code is flawless and bulletproof, you could just write small, granular programs and not need unit testing.

The idea of unit testing is to test the most granular things, then stack together tests to prove the larger case. If you're writing large tests, you lose a bit of the benefits there, although it's probably quicker to write larger tests.

Answer 3

There are two ways to help ensure the correctness of your unit tests:

• TDD: Write the test first, then write the code it's meant to test. That means you get to see them fail. If you know that it detects at least some classes of bugs (such as "I haven't implemented any functionality in the function I want to test yet"), then you know that it's not completely useless. It may still let some other bugs slip past, but we know that the test is not completely incorrect.
• Have lots of tests. If one test lets some bugs slip past, they'll most likely cause errors further down the line, causing other tests to fail. As you notice that, and fix the offending code, you get a chance to examine why the first test didn't catch the error as expected.

And finally, of course, keep the unit tests so simple that they're unlikely to contain bugs.

Answer 4

As above, the best way is to write the test before the actual code. Find real life examples of the code your testing also if applicable (mathematical formula or similar), and compare the unit test and expected output to that.

Answer 5

1. The complexity of the unit test code is (or should be) less (often orders of magnitude less) than the real code
2. The chance of your coding a bug in your unit test that exactly matches a bug in your real code is much less than just coding the bug in your real code (if you code a bug in your unit test that doesn't match a bug in your real code it should fail). Of course if you have made incorrect assumptions in your real code you are likely to make the same assumption again - although the mind set of unit testing should still reduce even that case
3. As already alluded to, when you write a unit test you have (or should have) a different mind set. When writing real code you're thinking "how do I solve this problem". When writing a unit test you're thinking, "how do I test every possibly way this could break"

As others have already said, it's not about whether you can prove that the unit tests are correct and complete (although that's almost certainly much easier with test code), as it is reducing the bug count to a very low number - and pushing it lower and lower.

Of course there has to come a point where your confident in your unit tests enough to rely on them - for example when doing refactorings. Reaching this point is usually just a case of experience and intuition (although there are code coverage tools that help).

Answer 6

This is something that bugs everyone that uses unit tests. If I would have to give you a short answer I 'd tell you to always trust your unit tests. But I would say that this should be backed up with your previous experience:

• Did you have any defects that were reported from manual testing and the unit test didn't catch (although it was responsible to) because there was a bug in your test?
• Did you have false negatives in the past?
• Are your unit tests simple enough?
• Do you write them before new code or at least in parallel?