Server mode GC seems to never collect Gen 0 Heap

Question

Clarified question (tl;dr)

After reading and profiling with all the results covered below, the problem seems to boil down to the GC not collecting G

Answer 1

You can try to enable gcTrimCommitOnLowMemory setting in Aspnet.config file in the .NET Framework directory:

When the gcTrimCommitOnLowMemory setting is enabled, the garbage collector evaluates the system memory load and enters a trimming mode when the load reaches 90%. It maintains the trimming mode until the load drops under 85%.

https://msdn.microsoft.com/en-us/library/bb384209(v=vs.110).aspx

Another option (since .net v4.5) is to set performanceScenario to "HighDensityWebHosting" in the same Aspnet.config file. This is useful for shared hosting scenarios as it will "tune garbage collection to optimize for memory": http://www.asp.net/aspnet/overview/aspnet-and-visual-studio-2012/whats-new#_Toc_perf_5

As you can see from CoreCLR sources the HighDensityWebHosting option mostly disables gcServer and gcConcurrent settings, but enables gcTrimCommitOnLowMemory: https://github.com/dotnet/coreclr/blob/cbf46fb0b6a0b209ed1caf4a680910b383e68cba/src/vm/perfdefaults.cpp

Answer 2

Not a direct answer, more like a bandaid, but if you can run .Net 4.5.1 Code in that w3wp process, you can compact the LOH and a lot of that unused allocated memory [might] reduce.

You could create App Start code that starts a timer that runs this every so often from inside the w3wp.exe process.

GCSettings.LargeObjectHeapCompactionMode = GCLargeObjectHeapCompactionMode.CompactOnce;

GC.Collect(); 

This feature was not added until 4.5.1 though, so you can't use it in a .Net Assembly that is not targetting at least 4.5.1 of the framework.

This might allow you to get rid of the web.config changes you made and keep the unallocated memory from staying high when it is not needed.

Answer 3

After much research, reading, and profiling I have been able to prove that our IIS memory usage is actually within the standard; this was done using the SysInternals Test Limit utility to push the server's physical memory usage to near max, once this was done all of our applications released their memory.

We do still have some kind of memory issue in our test environment that I need to investigate but at this point I think I can confidently say that this is entirely unrelated.

Moral of the story is to not assume that the cause reported with the issue is correct.