Pro's and Con's of unit testing after the fact

Question

I have a largish complex app around 27k lines. Its essentially a rule drive multithreaded processing engine, without giving too much away Its been partially tested as it\'s been

Answer 1

Pro post facto unit testing:

• Get documentation you can trust.
• Improve understanding of the code.
• Push toward refactoring and improving the code itself.
• Fix bugs that lurk in the code.

Con post facto unit testing:

• Waste time fixing bugs you can live with. (If you wrote 27KLOC, we hope it does something, right?)
• Spend time understanding and refactoring code you don't need to understand.
• Lose time that could go into the next project.

The unasked question is just how important an asset is this code to your organization, long term? The answer to this question determines how much you should invest. I have plenty of (successful) competitors where the major purpose of their code is to get out numbers to evaluate some new technique or idea. Once they have the numbers, the code is of little marginal value. They (rightly) test very carefully to make sure the numbers are meaningful. After that, if there are fifty open bugs that don't affect the numbers, they don't care. And why should they? The code has served its purpose.

Answer 2

There are many reasons to unit test code. The main reason I would advocate unit testing after the fact is simple. Your code is broken, you just don't know it yet.

There is a very simple rule in software. If the code is not tested, it's broken. This may not be immediately obvious at first, but as you begin testing, you will find bugs. It's up to you to determine how much you care about finding these bugs.

Besides this, there are several other important benefits of unit testing,

• regression testing will be made simpler
• other developers, that are less knowledgeable, can't break your desired behavior
• the tests are a form of self documentation
• can reduce time in future modifications (no more manual testing?, less bugs?)

The list can go on and on. The only real drawback is the time it takes to write these tests. I believe that drawback will always be offset by the time it takes you to debug problems you could have found while unit testing!

Answer 3

This is one of these difficult value judgement types of questions.

I would mostly agree with Epaga, that writing new tests as you fix bugs (perhaps with a couple of extra tests thrown in) is a good approach.

I would add two further comments:

• Doing backed-off black box testing to a unit before making large changes can be a good idea
• Consistency testing isn't unit testing, but certain types of program lend themselves to the easy generation of consistency tests. This might be one approach to making sure you don't break things.

Answer 4

Depending on how many bugs "manual testing" turns up, you could simply do test-driven bug fixing which in my experience is far more effective than simply driving up code coverage by writing "post-mortem" unit tests.

(Which is not to say writing unit tests afterwards is a bad idea, it's just that TDD is almost always a better idea.)

Answer 5

Unit testing is still definitely useful. Check out http://en.wikipedia.org/wiki/Unit_testing for a full list and explanation of the benefits.

The main benefits you will gain are documentation, making change easier, and it simplifies future integration.

There are really no costs to adding unit testing except your time. Realize though that the time you spend adding unit testing will reduce the amount of time you will need to spend in other areas of development by at least the same amount and most likely more.

Answer 6

I think one of the biggest con of testing "after the fact" is that you will probably have a harder time testing. If you write code without tests, you usually don't have testability in mind and end up writing code that is hard to test.

But, after you spent this extra time writing tests and changing your code for better testability, you'll be much more confident about making changes, once you won't need a lot of time to debug and check if everything is ok.

Finally, you might find new bugs which weren't caught before, and spend some time fixing it. But hey, that's what tests are for =)