Bug Hunting Strategies?

Question

Let\'s say you have a bug that was found in functional testing of a fairly complex part of the software. It could stem from bad/unexpected data in the database, middle-tier cod

Answer 1

1. Reproduce the bug in a debug environment.
2. Examine system state at the point the bug occurs to find the inconsistent / incorrect / unexpected elements of state that directly, visibly led to the bug occurring. Often, just eyeballing the code and call stack will immediately tell you what the problem is.
3. Add tests to all points where this state can be created / mutated within the normal flow of control.
4. Treating failures of these tests as a new bug, return to step two.

Rinse, lather, repeat until initial cause of the problem is found. Tedious and mechanical, but will get you there.

Except... occasionally the tests in an iteration of step 3 don't fail; most commonly, because some unrelated system corrupting memory is what is leading to the invalid state, or because the problem is threading/timing/uninitialised data dependent and introducing the tests alters timings and/or data layout sufficiently to alter or hide the symptoms. At this point, for this poster at least, the process becomes a more intuitive one, alternating between replacing sanity tests with less intrusive forms and selectively disabling modules to rule out sources of corruption.