eclipse memory analyzer sees small part (363,2MB) of entire heap dump (8GB)

Question

I was trying to investigate java.lang.OutOfMemoryError: GC limit exceeded which occurs at high load of our web app deployed in tomcat. Heap size was set to 8GB

Answer 1

MAT does not display the unreachable objects by default.

You can enable the option by going to Preferences -> Memory Analyzer -> Keep Unreachable Objects. Load the heap again once the option is enabled.

It will show the complete heap once the option is enabled. Even I was in same situation and was not able to get much information online and my manager showed me the option.Hope it helps.

Answer 2

We found a bug in MAT recently where it would only see a subset of the heap: https://bugs.eclipse.org/bugs/show_bug.cgi?id=404679#c3

The problem was that the JVM wrote a single HEAP_DUMP record that was more than 4GB and therefore the length in the header wrapped.

What version of MAT are you running? Please try a more recent build.

Answer 3

This is a usual behavior of MAT when collecting a heap dump of such a large heap. I frequently collect heap dumps of 8GB heaps and usually get a MAT profile showing ~1GB of live objects.

The 45 minutes freeze is also expected. My explanation is that during the heap dump collection time several FullGC cycles occur which reduce the size of the actual heap dump acquired. But I have not found official explanation or reference documenting why there is such a big difference.

Further, see this reference - MAT Does Not Show the Complete Heap :

Symptom: When monitoring the memory usage interactively, the used heap size is much bigger than what MAT reports.

During the index creation, the Memory Analyzer removes unreachable objects because the various garbage collector algorithms tend to leave some garbage behind (if the object is too small, moving and re-assigning addresses is to expensive). This should, however, be no more than 3 to 4 percent. If you want to know what objects are removed, enable debug output as explained here: MemoryAnalyzer/FAQ#Enable_Debug_Output

Answer 4

Actually I don't recommend that you use the jmap utility when you are troubleshooting a java.lang.OutOfMemoryError: GC limit exceeded problem. I have seen problems with this approach while the JVM is trashing (excessive # of major collections).

Please try the following approach instead and see if you get better results from the MAT parsing & analysis process:

• Add the following JVM HotSpot parameter -XX:+HeapDumpOnOutOfMemoryError
• Replicate the problem again and wait for the OOM event. The heap dump generation should be much faster which will increase the value and validity of the data.
• The JVM will then generate a JVM Heap Dump (HPROF format) following the OOM event.
• Load the JVM Heap Dump again in MAT and see if you are getting better results e.g. bigger heap footprint.

Regards, P-H