SymEnumSymbols returns ERROR_SUCCESS but gives no results

Question

I'm attempting to enumerate symbols from a DLL that I have loaded. For those interested, this is part of the CPPCoverage project, and for some functionality I need symbol data.

Breakdown of the problem

When the process is started or a DLL is loaded, symbols need to be enumerated for some of the new functionality that has been planned.

Basically, a process is created, and dbghelp is used to get symbol information. Next, symbols are iterated using SymEnumSymbols. There are two moments when this happens:

1. When the process is started (CREATE_PROCESS_DEBUG_EVENT)
2. When a DLL is loaded (LOAD_DLL_DEBUG_EVENT)

Everything works fine during (2). However, symbols cannot be enumerated during (1).

Behavior is that everything works fine, until the SymEnumSymbols call. The return value tells me there's an error, but GetLastError returns SUCCESS. Also, the callback function isn't called.

To make it even more weird, a call to SymGetSymFromName does actually work.

Minimal test case

static BOOL CALLBACK EnumerateSymbols(
                          PSYMBOL_INFO pSymInfo, ULONG SymbolSize, PVOID UserContext)
{
    std::cout << "Symbol: " << pSymInfo->Name << std::endl;
    return TRUE;
}

void Test()
{
    SymSetOptions(SYMOPT_LOAD_ANYTHING);

    STARTUPINFO si;
    PROCESS_INFORMATION pi;
    ZeroMemory(&si, sizeof(si));
    si.cb = sizeof(si);
    ZeroMemory(&pi, sizeof(pi));

    auto str = "FullPathToSomeExeWithPDB.exe";
    auto result = CreateProcess(str, NULL, NULL, NULL, FALSE,
                                DEBUG_PROCESS, NULL, NULL, &si, &pi);
    if (result == 0)
    {
        auto err = GetLastError();
        std::cout << "Error running process: " << err << std::endl;
        return;
    }

    if (!SymInitialize(pi.hProcess, NULL, FALSE))
    {
        auto err = GetLastError();
        std::cout << "Symbol initialization failed: " << err << std::endl;
        return;
    }

    bool first = false;
    DEBUG_EVENT debugEvent = { 0 };
    while (!first)
    {
        if (!WaitForDebugEvent(&debugEvent, INFINITE))
        {
            auto err = GetLastError();
            std::cout << "Wait for debug event failed: " << err << std::endl;
            return;
        }
        if (debugEvent.dwDebugEventCode == CREATE_PROCESS_DEBUG_EVENT)
        {
            auto dllBase = SymLoadModuleEx(
                pi.hProcess,
                debugEvent.u.CreateProcessInfo.hFile,
                str,
                NULL,
                reinterpret_cast<DWORD64>(debugEvent.u.CreateProcessInfo.lpBaseOfImage),
                0,
                NULL,
                0);

            if (!dllBase)
            {
                auto err = GetLastError();
                std::cout << "Loading the module failed: " << err << std::endl;
                return;
            }

            if (!SymEnumSymbols(pi.hProcess, dllBase, NULL, EnumerateSymbols, nullptr))
            {
                auto err = GetLastError();
                std::cout << "Error: " << err << std::endl;
            }

            first = true;
        }
    } 
    // cleanup code is omitted
}

Answer 1

Brr, quite a stumper. I got a repro for this in VS2017, using a simple do-nothing target executable built from the Win32 Console project template. Nothing I tried could convince SymEnumSymbols() to enumerate any symbols. I next expanded on the code, also trapping the LOAD_DLL_DEBUG_EVENT notification:

if (debugEvent.dwDebugEventCode == LOAD_DLL_DEBUG_EVENT) {
    auto base = SymLoadModule64(pi.hProcess, debugEvent.u.LoadDll.hFile, NULL, NULL, NULL, 0);
    if (!base) {
        auto err = GetLastError();
        std::cout << err << std::endl;
     }
    else {
        CloseHandle(debugEvent.u.LoadDll.hFile);
        SymEnumSymbols(pi.hProcess, base, NULL, EnumerateSymbols, nullptr);
    }
}

Along with setting the symbol search path correctly in SymInitialize(), that worked just fine and properly listed the symbols in ntdll.dll etc.

Conclusion: there is something wrong with the PDB file

That paid-off. Microsoft has been tinkering with the PDB file generation, starting in VS2015. They added the /DEBUG:FASTLINK option. Note that the linked docs are misleading, it is also the default in VS2015. The resulting PDB file cannot be properly enumerated by the operating system's version of DbgHelp.dll. The GetLastError() code was quite misleading and I spent entirely too much time on it, I think it merely indicates "I successfully enumerated nothing". Note how this code is documented for other DbgHelp api functions like SymSetContext and SymLoadModuleEx.

In VS2015 use Project > Properties > Linker > Debug > Generate Debug Info = "Optimize for debugging (/DEBUG)".

In VS2017 use Project > Properties > Linker > Debug > Generate Debug Info = "Generate Debug Information optimized for sharing and publishing (/DEBUG:FULL)".

Emphasizing that these setting matter on the target project, not the debugger project. Ideally there would a DbgHelp.dll version that could read debug info from the fastlink version of the PDB as well. I could not find one, the ones that came along with SDK 8.1 and SDK 10 did not solve the problem. Yet another case of the DevDiv and Windows groups not working together.

Answer 2

After the comment from @SimonMournier I ran a lot of other tests. Eventually, I was able to figure out what the issue here is. As it turns out, the linker flag /DEBUG:FastLink in Visual Studio actually causes the issue.

After some google'ing I found this notice on the community forums: https://developercommunity.visualstudio.com/content/problem/4631/dia-sdk-still-doesnt-support-debugfastlink.html

[...] Windows debuggers team has been informed to build a new dbghelp.dll with VS 2017 PDB/DIA static libraries and the next public release of Windows SDK (or debugger kits) will contain dbghelp.dll that is able to deal with fastlink PDBs. The latest VS 2017 pre-release would install a new dbghelp.dll under VS installation directory that works with fastlink PDBs.

So, in short, it simply won't work with Visual Studio 2015, because DIA simply doesn't support it. When we're upgrading to VS2017, it'll be automatically fixed. Also, when linking with /DEBUG , everything will work out fine.