Solving random crashes

Question

I am getting random crashes on my C++ application, it may not crash for a month, and then crash 10 times in a hour, and sometimes it may crash on launch, while sometimes it may

Answer 1

Try Valgrind (it's free, open-source):

The Valgrind distribution currently includes six production-quality tools: a memory error detector, two thread error detectors, a cache and branch-prediction profiler, a call-graph generating cache profiler, and a heap profiler. It also includes two experimental tools: a heap/stack/global array overrun detector, and a SimPoint basic block vector generator. It runs on the following platforms: X86/Linux, AMD64/Linux, PPC32/Linux, PPC64/Linux, and X86/Darwin (Mac OS X).

Valgrind Frequently Asked Questions

The Memcheck part of the package is probably the place to start:

Memcheck is a memory error detector. It can detect the following problems that are common in C and C++ programs.

• Accessing memory you shouldn't, e.g. overrunning and underrunning heap blocks, overrunning the top of the stack, and accessing memory after it has been freed.

• Using undefined values, i.e. values that have not been initialised, or that have been derived from other undefined values.

• Incorrect freeing of heap memory, such as double-freeing heap blocks, or mismatched use of malloc/new/new[] versus free/delete/delete[]

• Overlapping src and dst pointers in memcpy and related functions.

• Memory leaks.

Answer 2

Where I work, crashing programs usually generates a core dump file that can be loaded in windbg.

We then have an image of the memory at the time the program crashed. There's nothing much you can do with it, but a least it gives you the last call stack. Once you know the function which crashed, you might then be able to track down the problem are at least you might reduce the problem to a more reproductible test-case.

Answer 3

Another basic check: Make sure you do a full rebuild of your project. If you've been tweaking various files (especially header files) and doing partial builds then things can get messy if your build dependencies aren't perfect. A full rebuild just removes that possibility.

Also for Windows check out Microsoft's Debugging tools for Windows, and particularly their gflags tool.

Answer 4

If all else fails (particularly if performance under the debugger is unacceptable), extensive logging. Start with the entry points -- is the app transactional? Log each transaction as it comes in. Log all the constructor calls for your key objects. Since the crash is so intermittent, log calls to all the functions that might not get called every day.

You'll at least start narrowing down where the crash could be.

Answer 5

You've already heard how to handle this under linux: inspect core dumps and run your code under valgrind. So your first step could be to find the errors under Linux and then check if they vanish under mingw. Since nobody did mention mudflap here, I'll be doing it: Use mudflap if your Linux distribution supplies it. mudflap helps you to catch pointer misuse and buffer overflows by tracking the information where a pointer is actually allowed to point to:

• http://gcc.gnu.org/wiki/Mudflap_Pointer_Debugging

And for Windows: There is a JIT debugger for mingw, called DrMingw:

• http://code.google.com/p/jrfonseca/wiki/DrMingw

Answer 6

You have probably made a memory error where you put some values to not allocated space somehow, it is a good reason for random crashes, for a long time noone tries to use that memory so there will be no errors, you can take a look the places where you allocate memory and check where you extensively use pointers. Other than this, as others pointed out you should use extensive logging, in both screen and files.