Unexpected page handling (also, VirtualLock = no op?)

Question

This morning I stumbled across a surprising number of page faults where I did not expect them. Yes, I probably should not worry, but it still strikes me odd, because in my u

Answer 1

I don't have enough reputation to comment, so I'll have to add this as an answer.

Note that this code puts the thread to sleep after VirtualLock. According to a 2007 post by Raymond Chen, the OS is free to page it all out of physical RAM at this point and until the thread wakes up again [...] I suspect Raymond's advice applied in 2007, but is no longer true in Win7.

What romkyns said has been confirmed by Raymond Chen in 2014. That is, when you lock memory with Virtual­Lock, it will remain locked even if all your threads are blocked. He also says the fact that pages remain locked, may be just an implementation detail and not contractual.

This is probably not the case, because according to msdn, it is contractual

Pages that a process has locked remain in physical memory until the process unlocks them or terminates. These pages are guaranteed not to be written to the pagefile while they are locked.

Answer 2

Technically VirtualLock is a hint, and so the OS is allowed to ignore it. It's backed by the NtLockVirtualMemory syscall which on Reactos/Wine is implemented as a no-op, however Windows does back the syscall with real work (MiLockVadRange).

VirtualLock isn't guarranteed to succeed. Calls to this function require the SE_LOCK_MEMORY_PRIVILEGE to work, and the addresses must fulfil security and quota restrictions. Additionally after a VirtualUnlock, the kernel is no longer obliged to keep your page in memory, so a page fault after that is a valid action.

And as Raymond Chen points out, when you unlock the memory it can formally release the page. This means that the next VirtualLock on the next page might obtain that very same page again, so when you touch the original page you'll still get a page-fault.

Answer 3

VirtualLock remains a no-op (fault-wise)

I tried to reproduce this, but it worked as one might expect. Running the example code shown at the bottom of this post:

• start application (523 page faults)
• adjust the working set size (21 page faults)
• VirtualAlloc with MEM_COMMIT 2500 MB of RAM (2 page faults)
• VirtualLock all of that (about 641,250 page faults)
• perform writes to all of this RAM in an infinite loop (zero page faults)

This all works pretty much as expected. 2500 MB of RAM is 640,000 pages. The numbers add up. Also, as far as the OS-wide RAM counters go, commit charge goes up at VirtualAlloc, while physical memory usage goes up at VirtualLock.

So VirtualLock is most definitely not a no-op on my Win7 x64 machine. If I don't do it, the page faults, as expected, shift to where I start writing to the RAM. They still total just over 640,000. Plus, the first time the memory is written to takes longer.

---

Rather, as it looks, each single accessed page is created upon faulting, even if it had been locked previously.

This is not wrong. There is no guarantee that accessing a locked-then-unlocked page won't fault. You lock it, it gets mapped to physical RAM. You unlock it, and it's free to be unmapped instantly, making a fault possible. You might hope it will stay mapped, but no guarantees...

For what it's worth, on my system with a few gigabytes of physical RAM free, it works the way you were hoping for: even if I follow my VirtualLock with an immediate VirtualUnlock and set the minimum working set size back to something small, no further page faults occur.

Here's what I did. I ran the test program (below) with and without the code that immediately unlocks the memory and restores a sensible minimum working set size, and then forced physical RAM to run out in each scenario. Before forcing low RAM, neither program gets any page faults. After forcing low RAM, the program that keeps the memory locked retains its huge working set and has no further page faults. The program that unlocked the memory, however, starts getting page faults.

This is easiest to observe if you suspend the process first, since otherwise the constant memory writes keep it all in the working set even if the memory isn't locked (obviously a desirable thing). But suspend the process, force low RAM, and watch the working set shrink only for the program that has unlocked the RAM. Resume the process, and witness an avalanche of page faults.

In other words, at least in Win7 x64 everything works exactly as you expected it to, using the code supplied below.

---

There are two problems with tampering the WS size, however. First, you're generally not meant to have a gigabyte of minimum working set in a process

Well... if you want to VirtualLock, you are already tampering with it. The only thing that SetProcessWorkingSetSize does is allow you to tamper with it. It doesn't degrade performance by itself; it's VirtualLock that does - but only if the system actually runs low on physical RAM.

---

Here's the complete program:

#include <stdio.h>
#include <tchar.h>
#include <Windows.h>
#include <iostream>

using namespace std;

int _tmain(int argc, _TCHAR* argv[])
{
    SIZE_T chunkSize = 2500LL * 1024LL * 1024LL; // 2,626,568,192 = 640,000 pages
    int sleep = 5000;

    Sleep(sleep);

    cout << "Setting working set size... ";
    if (!SetProcessWorkingSetSize(GetCurrentProcess(), chunkSize + 5001001L, chunkSize * 2))
        return -1;
    cout << "done" << endl;

    Sleep(sleep);

    cout << "VirtualAlloc... ";
    UINT8* data = (UINT8*) VirtualAlloc(NULL, chunkSize, MEM_COMMIT, PAGE_READWRITE);
    if (data == NULL)
        return -2;
    cout << "done" << endl;

    Sleep(sleep);

    cout << "VirtualLock... ";
    if (VirtualLock(data, chunkSize) == 0)
        return -3;
    //if (VirtualUnlock(data, chunkSize) == 0) // enable or disable to experiment with unlocks
    //    return -3;
    //if (!SetProcessWorkingSetSize(GetCurrentProcess(), 5001001L, chunkSize * 2))
    //    return -1;
    cout << "done" << endl;

    Sleep(sleep);

    cout << "Writes to the memory... ";
    while (true)
    {
        int* end = (int*) (data + chunkSize);
        for (int* d = (int*) data; d < end; d++)
            *d = (int) d;
        cout << "done ";
    }

    return 0;
}

Note that this code puts the thread to sleep after VirtualLock. According to a 2007 post by Raymond Chen, the OS is free to page it all out of physical RAM at this point and until the thread wakes up again. Note also that MSDN claims otherwise, saying that this memory will not be paged out, regardless of whether all threads are sleeping or not. On my system, they certainly remain in the physical RAM while the only thread is sleeping. I suspect Raymond's advice applied in 2007, but is no longer true in Win7.