G1GC long pause with initial-mark

Question

With j7u5, G1GC

"-Xms3200m -Xmx3200m -XX:+UseG1GC -XX:ParallelGCThreads=14 -XX:ConcGCThreads=4 -XX:MaxGCPauseMillis=40 -XX:NewRatio=2 -XX:SurvivorRatio=10 -XX:+PrintGC -XX:+PrintGCDateStamps"

for a given performance test, my application hits a long pause after 5-hour run predictably. Except this big one (and only), there are small initial-mark phases.

Any suggestions to figure out what is happening to this long pause and how to tune it to avoid such a long pause which affects the latency targets (percentiles 98%, 99.999%)?

2012-12-22T09:48:57.966+0000: [GC pause (young) 2436M->1460M(3200M), 0.0627090 secs]
2012-12-22T09:49:07.295+0000: [GC pause (young) 2458M->1481M(3200M), 0.0871760 secs]
2012-12-22T09:49:18.905+0000: [GC pause (young) 2479M->1503M(3200M), 0.0930280 secs]
2012-12-22T09:49:32.366+0000: [GC pause (young) 2501M->1524M(3200M), 0.0827900 secs]
2012-12-22T09:49:44.576+0000: [GC pause (young) (initial-mark) 2522M->1546M(3200M), 3.4979530 secs]
2012-12-22T09:49:48.074+0000: [GC concurrent-root-region-scan-start]
2012-12-22T09:49:48.079+0000: [GC concurrent-root-region-scan-end, 0.0056590]
2012-12-22T09:49:48.080+0000: [GC concurrent-mark-start]
2012-12-22T09:49:48.173+0000: [GC concurrent-mark-end, 0.0932560 sec]
2012-12-22T09:49:48.180+0000: [GC remark, 0.0470160 secs]
2012-12-22T09:49:48.232+0000: [GC cleanup 1585M->944M(3200M), 0.0180490 secs]
2012-12-22T09:49:48.251+0000: [GC concurrent-cleanup-start]
2012-12-22T09:49:48.255+0000: [GC concurrent-cleanup-end, 0.0047270]

Answer 1

for logging, use -XX:+PrintGCTimeStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime for better understanding of the problem and bottlenecks

Its recommended to use -XX:+UnlockExperimentalVMOptions -XX:+AggressiveOpts -XX:+DoEscapeAnalysis -XX:+UseCompressedOops options with G1GC (if these are supported by native machine/server & JDK version)

Also please use -XX:GCPauseIntervalMillis=VALUE with the combination of -XX:MaxGCPauseMillis=VALUE (It'll control your pauses, better to get combination of best fit values of both params by doing some R&D) BTW we are successfully using combination of values as -XX:MaxGCPauseMillis=400 -XX:GCPauseIntervalMillis=8000

The default value of -XX:NewRatio param is 40, so its better to have it either 40 or more than 40 e.g. -XX:NewRatio=50

Answer 2

The G1 GC is an adaptive garbage collector with defaults that enable it to work efficiently without modification. Refer to below question for more details.

Java 7 (JDK 7) garbage collection and documentation on G1

If you follow above recommendations recommended by oracle:

1. Remove below parameters

   -XX:SurvivorRatio=10 
   

2. MaxGCPauseMillis=40 is too much aggressive compared to default value of 200 ms. Have a reasonable pause time goal.

3. Change -XX:ParallelGCThreads=14 -XX:ConcGCThreads=4 as per recommendation depending on number of CPU cores in your server.