IMO one of the most powerful ways of debugging server code is Differential Debugging, that is comparing two log files.
It is especially useful with legacy code when the current developers don't know all areas of the code. It helps narrow the search area and the code that has to be analysed. It does require good logging/tracing to be present already.
This works when you have a use case that succeeds and one that fails. For example feature X used to work in version 3 of your product but now is broken in version 4, and no one knows why.
Compare the logs, use awk or sed to strip out the redundant differences (these scripts can be reused):
- Time stamps
- Thread IDs - sometimes filter on a particular thread
- paths
- etc
When you see the logs diverge, this usually indicates a wrong decisions was made just before.
I used this technique to replace a proprietary middleware solution in a legacy system with CORBA and ensure that I did not change the behaviour of the application logic. But it is very useful in lots of situations.