vstest.console.exe with /EnableCodeCoverage just “hangs”… how debug, and how to fix?

Question

I\'m using vstest.console.exe (VS2012) to run tests with /EnableCodeCoverage, and with a .runsettings that defines a \"Code Coverage\" DataCollector (see CodeCoverage.runsetting

Answer 1

The only clue I could find that eventually led to a solution was in the Event Viewer, specifically:

.NET Runtime version 2.0.50727.5477 - Failed to CoCreate profiler.

This did ultimately led me to a fix, which is to add:

<startup useLegacyV2RuntimeActivationPolicy="true">
  <supportedRuntime version="v4.0.30319" />
</startup>        

to vstest.executionengine.exe.config (if running 64bit) or vstest.executionengine.x86.exe.config (if running 32bit).

This works under both VS (when spawned by devenv.exe) and from the command line (when spawned by vstest.console.exe)

Here's some notes that got me to this solution:

The new DLLs that are referenced include some pretty old code built against .NET 2.0. After a lot of searching, I've pieced together that:

• vstest.executionengine.exe uses the .NET 4 profiling stuff to do the dynamic code coverage instrumentation
• the event viewer error message is an obscure clue that vstest.executionengine.exe is failing to kick off the .NET 2.0 profiler
• as noted above, the fix is to update the .config (that is, vstest.executionengine.exe.config).

It isn't possible to add this stuff to the app.config of the project containing the tests; it must go into the vstest.executionengine's config (because this is the exe actually running).

Other things I tried (none of which, ultimately, helped):

Diagnostics in the registry

Computer\HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\11.0\EnterpriseTools\QualityTools\Diagnostics

• EnableTracing DWORD = 1
• TraceLevel DWORD = 4

Diagnostics in the *.config files

for vstest.console.exe, vstest.discoveryengine.*.exe, vstest.executionengine.*.exe

C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow

added:

  <system.diagnostics>
    <switches>
      <add name="TpTraceLevel" value="4" />
    </switches>
  </system.diagnostics>

... this causes log files to be written to %TEMP%

Red herring: Tried setting environment variables to disable

- COR_ENABLE_PROFILING=0
- COMPLUS_ProfAPI_ProfilerCompatibilitySetting=DisableV2Profiler

... in an attempt to disable the .NET 2 profiler, but did nothing.

Further investigation (using procexp.exe) showed that vstest.executionengine.exe always sets COR_ENABLE_PROFILING=1 irrespective of env var.

Moreover, COR_PROFILER's set to guid

• for me, was COR_PROFILER={B19F184A-CC62-4137-9A6F-AF0F91730165}
• via regedit, HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID{B19F184A-CC62-4137-9A6F-AF0F91730165}\InprocServer32
• corresponds to "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\IntelliTrace\11.0.0\x64\Microsoft.IntelliTrace.Profiler.11.0.0.dll"

Looking at the event viewer I saw info messages that the .NET 4 profiler was successfully started:

.NET Runtime version 4.0.30319.18063 - The profiler was loaded successfully.  Profiler CLSID: '{b19f184a-cc62-4137-9a6f-af0f91730165}'. Process ID (decimal): 7700.  Message ID: [0x2507].

The log files in %TEMP% also suggested no issues.

Red herring: Tried to go the other way, and enable via env vars:

COMPLUS_ProfAPI_ProfilerCompatibilitySetting=EnableV2Profiler
and also explicitly set
COR_PROFILER={B19F184A-CC62-4137-9A6F-AF0F91730165}

made no difference either.

Other red herrings

• occasional messages in Event Viewer of form:

  engine::notify_process_attach failed with exception: Session "MTM_7d145e0c-1c26-44b0-89e5-acc448aaae6d" does not exist.

• vstest.discoveryengine.TpTrace.log... the error "AddProcess : Failed to AddProcess 5" also seems to be irrelevant.

  'System.EventHandler`1[Microsoft.VisualStudio.TestTools.Execution.SessionStartEventArgs]' to 'Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector' I, 2800, 11, 2014/07/01, 10:59:11.875, PCKMA0419\vstest.discoveryengine.exe, Started Vangaurd process with command line unregister /wildcard /session:MTM_* I, 2800, 11, 2014/07/01, 10:59:11.880, PCKMA0419\vstest.discoveryengine.exe, Add Vangaurd process to the project object W, 2800, 11, 2014/07/01, 10:59:11.882, PCKMA0419\vstest.discoveryengine.exe, AddProcess : Failed to AddProcess 5 I, 2800, 11, 2014/07/01, 10:59:11.882, PCKMA0419\vstest.discoveryengine.exe, Started Vangaurd process with command line collect /session:MTM_64f33307-c936-469e-b068-482ec0ea45cf /output:"C:\Users\danhaywood\AppData\Local\Temp\MTM_64f33307-c936-469e-b068-482ec0ea45cf\c44e78af-2475-4747-99f3-e0fc3ca41d51\DanHaywood_PCKMA0419 2014-07-01 10_59_11.coverage" /config:

"C:\Users\danhaywood\AppData\Local\Temp\MTM_64f33307-c936-469e-b068-482ec0ea45cf\CodeCoverage.config" ~~~

Blogs consulted along the way:

• What 'additional configuration' is necessary to reference a .NET 2.0 mixed mode assembly in a .NET 4.0 project?
• http://blogs.msdn.com/b/dougste/archive/2009/12/30/failed-to-cocreate-profiler.aspx
• http://social.msdn.microsoft.com/Forums/en-US/cf079584-54b0-44df-a157-620cc613fca6/failed-to-cocreate-profiler?forum=netfxtoolsdev
• http://blogs.msdn.com/b/davbr/archive/2007/12/11/debugging-your-profiler-i-activation.aspx
• http://bytes.com/topic/c-sharp/answers/813648-net-crashing
• http://social.msdn.microsoft.com/Forums/en-US/ab48e868-528d-4d40-b04f-b8a39ba8bf0c/vs2010-on-windows-2008-wrong-setting-of-corprofiler-environment-variable-in-target-process?forum=vstsprofiler
• http://msdn.microsoft.com/en-us/library/dd778910(v=vs.110).aspx
• http://blogs.msdn.com/b/davbr/archive/2009/05/26/run-your-v2-profiler-binary-on-clr-v4.aspx
• http://cvpcs.org/blog/2011-06-23/getting_ncover_and_nunit_to_play_nicely_with_.net_4.0
• Force NCover 1.5.8 to use v4 framework like testdriven.net does?