Debugging shell extension in Windows 7

Question

I'm trying to debug shell extension (IContextMenu) in Windows 7 with Visual C++ 2008. I have set DesktopProcess=1 in the registry and set host app to explorer.exe. But when I start the debugger, it launches explorer.exe and then detaches from the process. DllMain of the shell extension isn't called.

The same code with exactly the same settings launched in debugger without any problems in Windows XP + Visual C++ 2008.

Any thoughts how to debug the shell extension in Win7?

Answer 1

Try launching explorer and THEN attaching the debugger to it.

Answer 2

I've found a nice workflow that I think is the fastest way to rapidly iterate the code-build-test cycle when developing shell extensions. The following should work on any Windows version.

First prepare - set the start program of your shell extension project to be c:\windows\explorer.exe and also set it to be the start-up project.

Then, whenever you want to debug your shell extension perform the following steps:

• Click on the task bar and press Alt-F4 - this will bring up the shut down dialog
• Press Ctrl-Alt-Shift-Escape - this combination will close explorer.
• Use Alt-Tab to go back to Visual Studio and press F5 - explorer will now launch with the VS debugger attached to it from the very beginning.
• When done, just stop the debugger session. This will kill the debugged instance of explorer and will also automatically start a normal instance of it. This will also unlock the shell extension DLL so that you can build it again.

Caveat on Vista and 7 - be sure to run the Visual Studio that you use for debugging in non-Administrator mode (non-elevated), so that the explorer is started in its usual non-elevated mode.

Answer 3

You could try putting a DebugBreak() call in your code. This should launch the just-in-time debugger at the call and give you an idea of what is going on.

Answer 4

You should take a look at gflags.exe, part of the standard debugging tools sdk. It's got all the options you need to configure (global)flags for any process startup/services/heap/pool-tagging/stacktrace's-on-allocation etc...

Answer 5

Debugging Explorer.exe is usually overkill for extensions that operate in a shell view.

I personally use a little app I made that hosts an instance of IExplorerBrowser similar to this example. If your IContextMenu item is not the default item then you can just use Notepad.exe and its open file dialog...