I've used minidumps on many game projects over the years and they seem to have about a 50% chance of having a valid call stack. What can I do to make them have better call stacks?
I've tried putting the latest dbghelp.dll in the exe directory. That seems to help some.
Is Visual Studio 2008 or 2010 any better? (I'm still on VS 2005).
The code I use looks like this sample.
One thing you can do to improve the accuracy of call stacks found in dumps is to use a debugger other than Visual Studio -- specifically, use WinDbg or another tool that uses the "Windows Debugger" debugging engine found in dbgeng.dll (as opposed to the "Visual Studio Debugger" debugging engine that Visual Studio uses).
In our experience, WinDbg is 100% reliable in producing good call stacks from the same dumps where Visual Studio produces unusable or wildly inaccurate call stacks. From what I can tell, in cases where an unhandled exception is the source of the crash WinDbg automatically performs the tricky process of reconstructing/recovering the exception callstack, but Visual Studio does not (or cannot?). The two debuggers use different heuristics for interpreting stacks
WinDbg can be daunting at first, so here's my quick guide on how to make it easier or even avoid having to use it directly.
A Mere Mortal's Guide To Extracting Good Callstacks
These are ordered from "fastest/easiest" to "slowest/most cryptic to interpret".
Easiest Option: use DbgDiag from Microsoft
This is a little-known tool that automates a lot of analysis of common problems, and it's simple enough to give to non-programmers or even customers. It's fast and nearly foolproof, and has become my "go to" tool for quickly analyzing an incoming crash dump.
- Launch the "DebugDiag Analysis" application
- Select the "CrashHangAnalysis" checkbox on the main page
- Drag-and-drop your dump into the "Data files" pane on the main page
- Click "Start Analysis"
After a few seconds to a few minutes it will spit out a nice .mhtml file containing an analysis of the problem, info about all the related thread, complete call stacks, etc. All hyperlinked and easy to use.DebugDiag even automates some of the more complicated analysis that is possible but painful in WinDbg (like tracking down which of the 350 threads in your application is responsible for a deadlock).
Note: Chrome will not download or open .mhtml files for security reasons, so you must open in Internet Explorer or Microsoft Edge for it to be usable. This is annoying, and I've filed a request with the DebugDiag team (dbgdiag@microsoft.com) to change the format to plain HTML
Middle option: Install WinDbg as an alternate debugging engine for Visual Studio
- Install Visual Studio if it's not yet installed. This needs to be done before the next step.
- Install the Windows Driver Kit (WDK)
- Launch Visual Studio, and (this part is important!) use the new "File -> Open -> Crash Dump..." option to open the dump. This will debug the crash dump using the Windows Debugger (if you instead drag-and-drop the dump on Visual Studio or use the standard "File -> Open -> File..." option to open the dump, it will debug it using the old Visual Studio debugging engine... so be careful to use the right option).
- You should now be able to see the correct call stack and navigate around using the Visual Studio GUI, although some things work differently (the watch windows require using the unfamiliar WinDbg syntax, thread IDs are different, etc). Note: the Visual Studio UI may be very sluggish, especially if many threads are involved and the 'threads' or 'parallel stacks' windows are open.
Hardcore option: Use WinDbg directly
- Launch WinDbg.exe
- Drag-and-drop your dump into the WinDbg window
- Type
!analyze -vand press Enter. After a little bit of time WinDbg will spit out a crash call stack, and also its estimation of what the source of the problem is. If you're analyzing a deadlock, you can try!analyze -v -hangand WinDbg will often show you the dependency chain involved.
At this point you may have all the info you need! However, if you then want to examine the process state in the Visual Studio debugger you can take the following additional steps:- Open the crash dump in Visual Studio
- Right-click in the callstack window and choose "Go to Disassembly"
- Paste the hex address from the top line of WinDbg's output callstack into the "Address" bar of the Disassembly window and press enter. You're now at the location of the crash, looking at the disassembled code.
- Right-click in the disassembly window and choose "Go To Source Code" to go to the source code for the location. Now you're looking at the source code at the crash site.
Note: all of the above require having correct symbol server paths configured, otherwise you won't be able to resolve the symbols in the call stacks. I recommend setting the _NT_SYMBOL_PATH environment variable so that it's automatically available to Visual Studio, WinDbg, and DebugDiag.
What's missing from your callstack? Do you have a bunch of addresses that don't resolve to valid function names (ie, 0x8732ae00 instead of CFoo:Bar())? If so, then what you need is to put your .PDBs where your debugger can find them, or set up a symbol server and set the "Symbol Paths" in the right-click context menu of the Modules pane.
We store every .PDB from every binary every time someone checks in a new Perforce changelist, so that when a dump comes back from anyone inside the office or any customer at retail, we have the .PDB corresponding to the version of the game they were running. With the symbol server and paths set, all I have to do is just double-click the .mdmp and it works every time.
Or do you have a call stack that appears to only have one function in it? Like, 0x8538cf00 without anything else above it in the stack? If so, then your crash is actually the stack itself being corrupted. If the return addresses in the backchain have been overwritten, naturally the debugger will be unable to resolve them.
Sometimes also you'll find that the thread that actually emits the minidump is not the one that threw the exception that caused the crash. Look in the Threads window to see if one of the other threads has the offending code in it.
If you are debugging a "Release" build -- that is to say, one compiled with all optimization flags turned on -- you will have to live with the fact that the debugger will have trouble finding local variables and some other data. This is because turning on optimizations means allowing the compiler to keep data on registers, collapse calculations, and generally do a variety of things that prevents data from ever actually being written to the stack. If this is your problem then you'll need to open up the disassembly window and chase the data by hand, or rebuild a debug binary and reproduce the problem where you can look at it.
Turn off Frame Pointer Optimization, if you need stack dumps. Frame pointers are used to explicitly define stack frames. Without them, the debugger has to deduce the location of each frame.
The code to record the minidump is unlikely to be relevant. The main things that a minidump records are module information (for getting symbols) and the full content of all thread stacks. Beyond that basic information (which is always recorded) nothing else matters.
Getting good symbols (including PE files) is crucial for stack walking. More details can be found here: https://randomascii.wordpress.com/2013/03/09/symbols-the-microsoft-way/
I find that Visual Studio is usually reliable at displaying call stacks. It automatically displays the relevant call stack from the exception record, and it makes changing threads easy so that you can see the call stacks of all threads. It does sometimes try to 'hide' details that it thinks might confuse you - whether that is good or bad depends on your skill level.
Windbg defaults to showing the call stack of the code that recorded the crash dump rather than the crashing call stack. Windbg requires that you go ".ecxr" or "!analyze -v" in order to see the crash stack. I find this annoying. Windbg also requires more configuration in order to be useful.
The two debuggers do have different stack walking heuristics. These heuristics are needed, for instance, if you call or return to address zero since there is no unwind information for that address. For 'clean' crashes where the failing instruction is in normal code these heuristics are less important.
The stack walking has almost certainly improved in the last ten years. VS 2015 Community Edition is very capable and is free so you might as well try it.
If you use windbg then you can try some experiments:
!vc7fpo - toggles some of the windbg heuristics.
!stackdbg d, 7, f - turns on windbg stack walk
k1 - walks one level of the stack, spitting diagnostics as controlled by !stackdbg
dds esp - dumps the raw contents of the stack, doing a symbol lookup on each pointer
If you upgrade to VS 2015 and still have problems then it is likely that the stack walking failures are specific to the crashes you are seeing. If a buffer overrun tromps the stack before crashing then the call stack will be irrevocably damaged. Your question has too little information about what failures you are seeing to give a definitive diagnosis. I find the stack displays of both debuggers fairly reliable, but I also usually understand why they sometimes fail and when that happens I can still extract the information that I need.
I dont use minidumps, but rather dump teh stack by "hand" into a logfile (see www.ddj.com/cpp/185300443 and How to Log Stack Frames with Windows x64).
I encounter a similar behavior like you do: Sometimes there is a valid call stack, sometimes there is not. In a minor number of cases the stack might be really corrupted. In maybe 1/3 of all cases the installed Exception handler is not called at all! I guess that its somehow a problem of the windows structured exception handling.
来源：https://stackoverflow.com/questions/1552788/why-dont-minidumps-give-good-call-stacks