How to prevent flushing to disk of a memory map opened on a windows temporary delete-on-close file

Question

UPDATE 2 / TL;DR

Is there some way to prevent dirty pages from a windows temporary delete-on-close file being flushed as a result of cl

Answer 1

After the bounty period expired without any answers that provided more insight or solved the mentioned problem, I decided to dig a little deeper and experiment some more with several combinations and sequences of operations.

As a result, I believe I have found a way to achieve memory maps shared between processes over temporary, delete-on-close files, that are not being flushed to disk when they are closed.

The basic idea involves creating the memory map when a temp file is newly created with a map name that can be used in a call to OpenFileMapping:

// build a unique map name from the file name.
auto map_name = make_map_name(file_name); 

// Open or create the mapped file.
auto mh = ::OpenFileMappingA(FILE_MAP_ALL_ACCESS, false, map_name.c_str());
if (mh == 0 || mh == INVALID_HANDLE_VALUE) {
    // existing map could not be opened, create the file.
    auto fh = ::CreateFileA(name.c_str(), kReadWrite, kFileSharing, nullptr, CREATE_NEW, kTempFlags, 0);
    if (fh != INVALID_HANDLE_VALUE) {
        // set its size.
        LARGE_INTEGER newpos;
        newpos.QuadPart = desired_size;
        ::SetFilePointerEx(fh, newpos, 0, FILE_BEGIN);
        ::SetEndOfFile(fh);
        // create the map
        mh = ::CreateFileMappingA(mh, nullptr, PAGE_READWRITE, 0, 0, map_name.c_str());
        // close the file handle
        // from now on there will be no accesses using file handles.
        ::CloseHandle(fh);
    }
}

Thus, the file handle is only used when the file is newly created, and closed immediately after the map is created, while the map handle itself remains open, to allow opening the mapping without requiring access to a file handle. Note that a race condition exists here, that we would need to deal with in any "real code" (as well as adding decent error checking and handling).

So if we got a valid map handle, we can create the view:

auto map_ptr = MapViewOfFile(mh, FILE_MAP_ALL_ACCESS, 0, 0, 0);
if (map_ptr) {
    // determine its size.
    MEMORY_BASIC_INFORMATION mbi;
    if (::VirtualQuery(map_ptr, &mbi, sizeof(MEMORY_BASIC_INFORMATION)) > 0) 
        map_size = mbi.RegionSize;
}

When, some time later closing a mapped file: close the map handle before unmapping the view:

if (mh == 0 || mh == INVALID_HANDLE_VALUE) {
    ::CloseHandle(mh);
    mh = INVALID_HANDLE_VALUE;
}
if (map_ptr) {
    ::UnmapViewOfFile(map_ptr);
    map_ptr = 0;
    map_size = 0;
}

And, according to the test I have performed so far, this does not cause flushing dirty pages to disk on close, problem solved. Well partially anyway, there may still be a cross-session map name sharing issue.

Answer 2

If I take it correctly, commenting out Arena2 part of code shall reproduce the issue without the need for second process. I have tried this:

1. I edited base_path as follows for convenience:

   char base_path[MAX_PATH];
   GetTempPathA(MAX_PATH, base_path);
   strcat_s(base_path, MAX_PATH, "page_pool");
   

2. I edited n_pages = 1536 * 128 to bring the used memory to 1.5GB, compared to your ~800mb.
3. I have tested TempFileMapping(false) and TempFileMapping(true), one at a time, for the same results.
4. I have tested with Arena2 commented out and intact, for the same results.
5. I have tested on Win8.1 x64 and Win7 x64, for ±10% same results.
6. In my tests, code runs in 2400ms ±10%, only 500ms ±10% spent on deallocating. That's clearly not enough for a flush of 1.5GB on a low-spinning silent HDDs I have there.

So, the question is, what are you observing? I'd suggest that you:

1. Provide your times for comparison
2. Use a different computer for tests, paying attention to excluding software issues such as "same antivirus"
3. Verify that you're not experiencing a RAM shortage.
4. Use xperf to see what's happening during the freeze.

Update I have tested on yet another Win7 x64, and times are 890ms full, 430ms spent on dealloc. I have looked into your results, and what is VERY suspicious is that almost exactly 4000ms is spent in freeze each time on your machine. That can't be a mere coincidence, I believe. Also, it's rather obvious now the the problem is somehow bound to a specific machine you're using. So my suggestions are:

1. As stated above, test on another computer yourself
2. As stated above, Use XPerf, it will allow you to see what exactly happens in user mode and kernel mode during the freeze (I really suspect some non-standard driver in the middle)
3. Play with number of pages and see how it affects the freeze length.
4. Try to store files on a different disk drive on the same computer where you have tested initially.