I have an odd situation I am trying to figure out.
The Genesis:
I am running my program on a physical machine with 16 cores and
From vague memory and having a read through: http://msdn.microsoft.com/en-us/library/ee787088.aspx, I think one trigger of a Gen 2 GC can be a Gen 2 segment filling up. The article states that Server GC uses larger segments, so as already noted, this is probably important for your performance.
Having the machine wait until it has virtually no memory free will mean you get one hell of a GC at some stage. This is probably not ideal. If your time in GC is so high, it's a sign you're allocating too many objects that are survivng long enough to get past gen 0 & 1, and doing it in a repititive way. If the memory usage of your application is not rising over time, this indicates that these objects are actually short lived, but live long enough to survive a 0 and a 1 collection. This is a bad situation - you're allocating a short lived object but paying a full Gen 2 collection cost to clean it up.
If that's the case, you have a few different directions to take:
It's likely that a combination of all of these would be a good solution. You need to have a good understanding of what objects you're allocating, how long they're living, and how long they actually need to live to fulfill your purpose.
The GC is happy with temporary objects that have short lifetimes (as in are collectable by the GC quickly), or long term/permanent obects that have long lifetimes. Allocating lots of objects in the middle of those two categories is where you get the pain. So allocate less of them or change their lifetimes to match their usage scenario.