How can I guarantee catching a EXCEPTION_STACK_OVERFLOW structured exception in C++ under Visual Studio 2005?

Question

Background

• I have an application with a Poof-Crash[1]. I'm fairly certain it is due to a blown stack.
• The application is Multi-Threaded.
• I am compiling with "Enable C++ Exceptions: Yes With SEH Exceptions (/EHa)".
• I have written an SE Translator function and called _set_se_translator() with it.
• I have written functions for and setup set_terminate() and set_unexpected().
• To get the Stack Overflow, I must run in release mode, under heavy load, for several days. Running under a debugger is not an option as the application can't perform fast enough to achieve the runtime necessary to see the issue.
• I can simulate the issue by adding infinite recursion on execution of one of the functions, and thus test the catching of the EXCEPTION_STACK_OVERFLOW exception.
• I have WinDBG setup as the crash dump program, and get good information for all other crash issues but not this one. The crash dump will only contain one thread, which is 'Sleep()'ing. All other threads have exited.

The Question

None of the things I've tried has resulted in picking up the EXCEPTION_STACK_OVERFLOW exception.

Does anyone know how to guarantee getting a a chance at this exception during runtime in release mode?

Definitions

1. Poof-Crash: The application crashes by going "poof" and disappearing without a trace.

(Considering the name of this site, I'm kind of surprised this question isn't on here already!)

Notes

1. An answer was posted briefly about adjusting the stack size to potentially force the issue sooner and allow catching it with a debugger. That is a clever thought, but unfortunately, I don't believe it would help. The issue is likely caused by a corner case leading to infinite recursion. Shortening the stack would not expose the issue any sooner and would likely cause an unrelated crash in validly deep code. Nice idea though, and thanks for posting it, even if you did remove it.

Answer 1

Everything prior to windows xp would not (or would be harder) generally be able to trap stack overflows. With the advent of xp, you can set vectored exception handler that gets a chance at stack overflow prior to any stack-based (structured exception) handlers (this is being the very reason - structured exception handlers are stack-based).

But there's really not much you can do even if you're able to trap such an exception.

In his blog, cbrumme (sorry, do not have his/her real name) discusses a stack page neighboring the guard page (the one, that generates the stack overflow) that can potentially be used for backout. If you can squeeze your backout code to use just one stack page - you can free as much as your logic allows. Otherwise, the application is pretty much dead upon encountering stack overflow. The only other reasonable thing to do, having trapped it, is to write a dump file for later debugging.

Hope, it helps.

Answer 2

I'm not convinced that you're on the right track in diagnosing this as a stack overflow.

But in any case, the fact that you're getting a poof!, plus what you're seeing in WinDbg

The crash dump will only contain one thread, which is 'Sleep()'ing. All other threads have exited.

suggests to me that somebody has called the C RTL exit() function, or possibly called the Windows API TerminateProcess() directly. That could have something to do with your interrupt handlers or not. Maybe something in the exception handling logic has a re-entrance check and arbitrarily decides to exit() if it's reentered.

My suggestion is to patch your executables to put maybe an INT 3 debug at the entry point to exit (), if it's statically linked, or if it's dynamically linked, patch up the import and also patch up any imports of kernel32::TerminateProcess to throw a DebugBreak() instead.

Of course, exit() and/or TerminateProcess() may be called on a normal shutdown, too, so you'll have to filter out the false alarms, but if you can get the call stack for the case where it's just about to go proof, you should have what you need.

EDIT ADD: Just simply writing your own version of exit() and linking it in instead of the CRTL version might do the trick.

Answer 3

I remember code from a previous workplace that sounded similar having explicit bounds checks on the stack pointer and throwing an exception manually.

It's been a while since I've touched C++ though, and even when I did touch it I didn't know what I was doing, so caveat implementor about portability/reliability of said advice.

Answer 4

Have you considered ADPlus from Debugging Tools for Windows?

ADPlus attaches the CDB debugger to a process in "crash" mode and will generate crash dumps for most exceptions the process generates. Basically, you run "ADPlus -crash -p yourPIDhere", it performs an invasive attach and begins logging.

Given your comment above about running under a debugger, I just wanted to add that CDB adds virtually zero overhead in -crash mode on a decent (dual-core, 2GB RAM) machine, so don't let that hold you back from trying it.

Answer 5

You can generate debugging symbols without disabling optimizations. In fact, you should be doing that anyways. It just makes debugging harder.

And the documentation for _set_se_translator says that each thread has its own SE translator. Are you setting one for each thread?

set_unexpected is probably a no-op, at least according to the VS 2005 documentation. And each thread also has its own terminate handler, so you should install that per thread as well.

I would also strongly recommend NOT using SE translation. It takes hardware exceptions that you shouldn't ignore (i.e., you should really log an error and terminate) and turns them into something you can ignore (C++ exceptions). If you want to catch this kind of error, use a __try/__except handler.