Is there a way to expand the current WorkingSet of a process to 1GB? [duplicate]

Answer 1

The only way that we could find to increase the WorkingSet of a process under .NET, to reduce soft page faults, was to run the entire .NET application under a custom CLR Host. This is a non-trivial exercise, requiring about 800 lines of custom written, rather dense C++ code. The C++ code intercepts the .NET calls to the Win32 memory management methods, altering the behavior of the .NET runtime so it doesn't free memory as aggressively as it normally would.

This has the effect of incurring all of the soft page faults when the application starts up, so that during normal application execution, the number of soft page faults in the .NET app drops pretty much to zero.

This means that the application may be memory hungry, but it runs faster. In other words, we are sacrificing memory usage for increased realtime performance.

Answer 2

I think assigning the process to a Job Object and setting JOB_OBJECT_LIMIT_WORKINGSET and MinimumWorkingSetSize might work.

Answer 3

I think there is a misunderstanding here: The GC freeing unused memory is good. It is of no use to keep it around just to see a higher number in the WS metric... You don't want unused memory in the WS. You are optimizing for the wrong metric.

The number of page faults is not that meaningful because there are soft and hard faults. Hard faults read from disk. Soft faults mean nothing in most cases. They are an implementation detail of the OS. For example, for every newly allocated page you touch a soft fault occurs.

Here is some evidence that your red graph is not showing hard faults: At the end it shows about 500/sec. Your disk cannot deliver 500 IOPs so it cannot be hard faults.

You probably want to measure (and reduce) hard faults only. Only hard faults impact performance in a meaningful way.

Also, setting WS limits does not influence what the GC does. The GC is not an operating system component. Is is a user-mode library. It has nothing to do with what the OS decides to use as the working set. For that reason you cannot make the GC not release memory by setting WS limits.

So why does the WS shrink when the GC runs? Because the GC deletes some memory. Setting some WS limit cannot prevent deletion of memory.

In the comments it was suggested the GC might itself call some API to shrink the working set. I cannot see any reason at all why it would do that. Why would the GC force pages out of the process? Remember: WS and allocated pages are not the same (at all!). The GC might release memory. It does not force the WS to shrink. What purpose would that have?

Answer 4

To achieve what you want you need to call/pinvoke SetWorkingSetSizeEx with 1 GB as minimum (second param) and QUOTA_LIMITS_HARDWS_MIN_ENABLE as fourth param which makes sure that workingset size won't go below minimum you gave even in "high memory pressure" conditions of the system.

The system behaviour also depends on privileges of the caller, depending on OS version etc. you might need SE_INC_WORKING_SET_NAME and/or SE_INC_BASE_PRIORITY_NAME !

Another (nicer) option which uses these APIs "behind the scenes" is a .NET wrapper you can find here.

Answer 5

If your problem was that your process was getting its WS trimmed too agressively in low-memory situations, you could take care of it by calling SetProcessWorkingSetSize or just setting Process.CurrentProcess.MinWorkingSet.

What you've shown, though, is that your working set is getting reduced by a GC. This tells me that what's really happening is the GC is deallocating the pages that make up your WS. If this is the case, you have an address space problem rather than a working set problem, and there's no system call you can make to prevent it. Ideally you would be able to tell the GC not to return its memory to the OS, but .NET doesn't have a feature like that.

To solve an address space problem you will have to reuse objects you have already allocated. If your problem is with the Large Object Heap, it's most likely due to collections. For example, rather than creating a new array/list/dictionary, call its Clear method and reuse it. If your problem is strings, you might be able to get away with using StringBuilders to stay out of the LOH.

If you have certain types of object you create lots of, consider creating a pool of them that get recycled. I've never done such a thing, but if I were to implement it I would create an object with a static factory method that pulls objects out of a pool and calls an initializer instead of having public constructors, and put a finalizer on it that puts it back in the pool and nulls out any references in it. Depending on the needs, the pool might be a ConcurrentBag<WeakReference<T>>.