Not totally sure that I've got this problem nailed down yet, but here's what I'm seeing and what I think is going on.
I have a Win32 program mostly written in C that loads a C++ DLL. That DLL passes data from the C program to another application via a COM object--one that's probably instantiated by the DLL itself. All of this has apparently worked fine through at least Windows XP and Windows 7 (possibly Win95 and Win98, I'd need to look back deeper into the code history to find out when this interface was introduced), but in Windows 10 the program crashes during the FreeLibrary() call for this DLL.
While checking this in the debugger, DLL_DETACH_PROCESS seems to be handled successfully (no code is executed when handling that message). The crash occurs after (or while) leaving the code from the Entry Point.
If I continue to Step In, I end up in a header file called utilcls.h, which seems to be one of the Borland C Builder 6 header files. I believe the template code therein is related to the COM object being torn down. An Unbind() call passes, and this is the last line of code I can step to before the crash.
If I use the debugger's CPU window and continue stepping, everything that remains appears to be related to freeing memory before the crash comes, but it's quite a lot of CPU-stepping to get there.
The crash raises APPCRASH with exception 0xc0000602, referring back to Combase.dll.
Simply not calling FreeLibrary for that DLL allows the application to close successfully, but my assumption is that the FreeLibrary call is important.
The COM object is released by the data-sharing application prior to the FreeLibrary() call, which allows that application to close. My assumption at the moment is that some of this unlinking is happening differently in the newer operating system and this is causing the crash, but I don't know how to be find out for sure.
My question(s):
if it's obvious to someone else who better knows what they're doing, what causes this crash?
what are the next steps in trying to debug this? I've exhausted my knowledge of the debugging environment I'm using and don't know COM or DLLs well enough to know what the next question to ask is.
Some of the debugger output RbMm requested:
0:000:x86> t
ntdll_77b40000!RtlIsCriticalSectionLockedByThread+0x1b:
77b7256b c20400 ret 4
0:000:x86> t
combase!DecrementMTAUsageHelper+0x5b:
7527a2d6 85c0 test eax,eax
0:000:x86> r eax
eax=00000001
0:000:x86> t
combase!DecrementMTAUsageHelper+0x5d:
7527a2d8 0f859a000000 jne combase!DecrementMTAUsageHelper+0xfd (7527a378) [br=1]
0:000:x86> t
combase!DecrementMTAUsageHelper+0xfd:
7527a378 e89e9e0f00 call combase!CrashProcessWithWERReport (7537421b)
At that point, the stack looks roughly like this:
ChildEBP RetAddr Args to Child
0019f9b8 7527a37c 063f4248 753d8448 00000000 combase!CrashProcessWithWERReport+0x35
0019f9e8 75292bfc 753d8448 7529257e 00000000 combase!DecrementMTAUsageHelper+0x101
(Inline) -------- -------- -------- -------- combase!DecrementMTAUsage+0x9
0019f9f0 7529257e 00000000 00000000 00000000 combase!CDllHost::MTAUninitializeApartmentOnly+0xe
0019fa08 7527543a 00000000 063f4248 00712410 combase!CDllHost::ClientCleanupFinish+0x4d
0019fa30 75276361 00000000 0019fa8c 00000000 combase!DllHostProcessUninitialize+0xa0
0019fa58 7527a452 000d06f6 00712410 00000000 combase!ApartmentUninitialize+0xe4
0019fa70 752c2a1e 000d06f6 00712e18 00712e80 combase!wCoUninitialize+0xd0
0019fa94 74ed3e58 00000003 74c17ff1 a6d0e607 combase!CoUninitialize+0x7e
0019fa9c 74c17ff1 a6d0e607 000b0792 74ed48f0 imm32!CtfImmCoUninitialize+0x48
0019fb7c 74809ea6 00050004 000d06f6 00000000 msctf!TF_Notify+0x581
0019fb98 748080dc 00050004 000d06f6 00000000 user32!CtfHookProcWorker+0x36
0019fbe0 74807fa6 0019fc34 0019fc24 00000000 user32!CallHookWithSEH+0x5c
0019fc08 77bb0006 0019fc24 00000018 0019fc80 user32!__fnHkINDWORD+0x26
0019fc38 710623fb 000b0792 04ff11aa 05480e70 ntdll!KiUserCallbackDispatcher+0x36
0019fc50 050364e4 000b0792 050376d8 05480e70 apphelp!DWM8AND16BitHook_DestroyWindow+0x2b
0019fc8c 05051007 00000000 05055034 00000001 myDLL!myCOMObject_tlbFinalize+0x408a4
0019fcb4 050511c6 0019fcd0 00000001 04ff1318 myDLL!myCOMObject_tlbFinalize+0x5b3c7
0019fcd8 04ff13d3 05055034 77badcce 04ff0000 myDLL!myCOMObject_tlbFinalize+0x5b586
0019fd00 77b807c6 04ff1318 04ff0000 00000000 myDLL+0x13d3
0019fd50 77b6aa5e 00000000 00000000 259704e5 ntdll!LdrpCallInitRoutine+0x43
0019fdb8 77b6e6c8 00000000 0071dd60 00000000 ntdll!LdrpProcessDetachNode+0xbb
0019fdd8 77b6e5af 25970745 0071e560 c000022d ntdll!LdrpUnloadNode+0x100
0019fe18 77b6e4f6 004afcc4 004ae3a4 04ff0000 ntdll!LdrpDecrementModuleLoadCountEx+0xa7
0019fe38 746e9d56 04ff0000 006e33c5 00000000 ntdll!LdrUnloadDll+0x86
0019fe4c 0049261c 04ff0000 00000000 00493034 KERNELBASE!FreeLibrary+0x16
0019fe64 00441895 004afc98 fffffffe 0019fee8 rpopdbg!_GetExceptDLLinfo+0x914bf
Working on the rest now, but my guess is I need to figure out how to do the cleanup properly on the COM object? Maybe in response to DLL_DETACH_PROCESS?
The crash raises APPCRASH with exception 0xc0000602, referring back to Combase.dll
combase.dll used 0xc0000602 (STATUS_FAIL_FAST_EXCEPTION) code only from
void CrashProcessWithWERReport();
(which called RaiseFailFastException with this code)
CrashProcessWithWERReport called only from DecrementMTAUsageHelper on 2 conditions - CoDecrementMTAUsage called more times than CoIncrementMTAUsage or (and i almost sure in this reason) DecrementMTAUsageHelper called when calling thread hold Loader critical section - so while DLL loading or unloading process. from MSDN
Don't call CoDecrementMTAUsage during process shutdown or inside dllmain. You can call CoDecrementMTAUsage before the call to start the shutdown process.
so my guess - some code call CoDecrementMTAUsage in your DLL unloading process (when you call FreeLibrary)
your DLL can not direct call CoIncrementMTAUsage / CoDecrementMTAUsage because this new API, exist begin from win 8 (also check your code on win 8.1 - i think also will be crash), but this api can be indirect called from other system components.
i can assume that your DLL not direct free some used resources or you call FreeLibrary when DLL still holding some resources (so you call FreeLibrary without proper cleanup calls fro DLL) and as result this resources begin free (CoDecrementMTAUsage) in unloading process
what are the next steps in trying to debug this?
you need use debugging with symbols files (say with winDbg). set breakpoints at DecrementMTAUsageHelper, CoDecrementMTAUsage and may be CoIncrementMTAUsage - are i right that call toRtlIsCriticalSectionLockedByThread return TRUE (this api called from begin of DecrementMTAUsageHelper).
in any case post the thread call stack at DecrementMTAUsageHelper call point(just before crash) and possible on CoIncrementMTAUsage too
---------------------- EDIT -------------------------
by view stack trace visible that your DLL call DestroyWindow from DllMain.
apphelp!DWM8AND16BitHook_DestroyWindow
this is the bug just by 2 reasons - at first - read this article -
The thread that gets the DLL_PROCESS_DETACH notification is not necessarily the one that got the DLL_PROCESS_ATTACH notification. You can't do anything with thread affinity in your DLL_PROCESS_ATTACH or DLL_PROCESS_DETACH handler since you have no guarantee about which thread will be called upon to handle these process notifications. The classic example of this, which I'm told the Developer Support team run into with alarming frequency, is a DLL that creates a window in its DLL_PROCESS_ATTACH handler and destroys it in its DLL_PROCESS_DETACH handler.
but your crash here by another reason, not listed in article - DllMain have many restrictions, what can not be called inside it. despite DestroyWindow not direct listed here, but as show your case - this is illegal call (even if we called on same thread, on which this window was created) - while your window is destroyed imm32.CtfImmNotify(msctf!TF_Notify) is called
0019fa9c 74c17ff1 a6d0e607 000b0792 74ed48f0 imm32!CtfImmCoUninitialize+0x48
0019fb7c 74809ea6 00050004 000d06f6 00000000 msctf!TF_Notify+0x581
0019fb98 748080dc 00050004 000d06f6 00000000 user32!CtfHookProcWorker+0x36
0019fbe0 74807fa6 0019fc34 0019fc24 00000000 user32!CallHookWithSEH+0x5c
and as result CoUninitialize is called from DllMain !
from MSDN
do not call CoInitialize, CoInitializeEx, or CoUninitialize from the DllMain function.
here inside FINAL CoUninitialize called DecrementMTAUsage which determinate that we inside loader lock by call RtlIsCriticalSectionLockedByThread and CrashProcessWithWERReport called.
Solution ?
of course the best is fix DLL, but if this is impossible - think next "hack" will be work
HRESULT hr = CoInitialize(0); // asume that we in STA
FreeLibrary(hDLL);
if (0 <= hr) CoUninitialize();
with this CoUninitialize of course anyway will be called from imm32!CtfImmCoUninitialize but this will be NOT FINAL Uninitialize and as result DecrementMTAUsage will be not called
来源：https://stackoverflow.com/questions/41008037/debugging-a-crash-while-unloading-dll-in-win-10-but-not-win-7