Actually you have quite a lot of possibilities. Either with recompilation of source code or without.
With recompilation.
- Additional logging. Either into program's logs or using system logging (eg. OutputDebugString or Events Log on Windows). Also use following steps:
- Always include timestamp at least up to seconds resolution.
- Consider adding thread-id in case of multithreaded apps.
- Add some nice output of your structures
- Do not print out enums with just %d. Use some
ToString() or create some EnumToString() function (whatever suits your language)
- ... and beware: logging changes timings so in case of heavily multithreading you problems might disappear.
- More details on this here.
- Introduce more asserts
- Unit tests
- "Audio-visual" monitoring: if something happens do one of
- use buzzer
- play system sound
- flash some LED by enabling hardware GPIO line (only in embedded scenarios)
Without recompilation
- If your application uses network of any kind: Packet Sniffer or I will just choose for you: Wireshark
- If you use database: monitor queries send to database and database itself.
- Use virtual machines to test exactly the same OS/hardware setup as your system is running on.
- Use some kind of system calls monitor. This includes
- On Unix box strace or dtrace
- On Windows tools from former Sysinternals tools like http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx, ProcessExplorer and alike
- In case of Windows GUI stuff: check out Spy++ or for WPF Snoop (although second I didn't use)
- Consider using some profiling tools for your platform. It will give you overview on thing happening in your app.
- [Real hardcore] Hardware monitoring: use oscilloscope (aka O-Scope) to monitor signals on hardware lines
- Source code debugging: you sit down with your source code and just pretend with piece of paper and pencil that you are computer. Its so called code analysis or "on-my-eyes" debugging
- Source control debugging. Compare diffs of your code from time when "it" works and now. Bug might be somewhere there.
And some general tips in the end:
- Do not forget about Text to Columns and Pivot Table in Excel. Together with some text tools (
awk, grep or perl) give you incredible analysis pack. If you have more than 32K records consider using Access as data source.
- Basics of Data Warehousing might help. With simple cube you may analyse tons of temporal data in just few minutes.
- Dumping your application is worth mentioning. Either as a result of crash or just on regular basis
- Always generate you debug symbols (even for release builds).
- Almost last but not least: most mayor platforms has some sort of command line debugger always built in (even Windows!). With some tricks like conditional debugging and break-print-continue you can get pretty good result with obscure bugs
- And really last but not least: use your brain and question everything.
In general debugging is like science: you do not create it you discover it. Quite often its like looking for a murderer in a criminal case. So buy yourself a hat and never give up.
