Why does a .NET program survive a corrupt stack? (when using the wrong calling convention)

Question

In VS2010, the managed debugging assistant will give you a pInvokeStackImbalance exception (pInvokeStackImbalance MDA) if you call a function using the wrong calling convent

Answer 1

The runtime can detect the stack imbalance because the stack pointer isn't where it's expected. That is, in the case of StdCall, where the called function is expected to clean up the stack, then the runtime could do this:

SavedSP = SP; // save the stack pointer
// now push parameters
// call the external function.
if (SP != SavedSP)
{
    // error!
}

Now, if the value of SP is less than SavedSP, then there's extra stuff on the stack--meaning that the runtime can just go ahead and restore the saved stack pointer.

The runtime should always be able to detect a stack imbalance. Whether or not it can always recover is unknown to me. But in the case of inadvertently calling a Cdecl method as StdCall, it should be able to recover without trouble, since there will be extra stuff on the stack that it can ignore.

As to why bother? As you say, the difference between StdCall and Cdecl is really only who's responsible for stack cleanup. Also, StdCall is not compatible with variable argument lists (i.e. printf in C), although I don't know if it's even possible to call such a method from .NET (haven't ever had a need to). In any case, although there doesn't appear to be a particular problem with calling a Cdecl method with StdCall, I kind of like knowing that there's a potential error. To me, it's like the error message that the compiler gives when you write:

uint x = 3;
int y = x;  // error!

I know that the assignment is okay, but the compiler disallows it because it's a potential source of bugs. In my mind, an unbalanced stack is a potential source of bugs. No, it is a bug that can cause some very bad things to happen. I'd rather the runtime told me about it so that I can fix the problem.