Java using much more memory than heap size (or size correctly Docker memory limit)

Question

For my application, the memory used by the Java process is much more than the heap size.

The system where the containers are running starts to have memory problem be

Answer 1

TL;DR

The detail usage of the memory is provided by Native Memory Tracking (NMT) details (mainly code metadata and garbage collector). In addition to that, the Java compiler and optimizer C1/C2 consume the memory not reported in the summary.

The memory footprint can be reduced using JVM flags (but there is impacts).

The Docker container sizing must be done through testing with the expected load the application.

---

Detail for each components

The shared class space can be disabled inside a container since the classes won't be shared by another JVM process. The following flag can be used. It will remove the shared class space (17MB).

-Xshare:off

The garbage collector serial has a minimal memory footprint at the cost of longer pause time during garbage collect processing (see Aleksey Shipilëv comparison between GC in one picture). It can be enabled with the following flag. It can save up to the GC space used (48MB).

-XX:+UseSerialGC

The C2 compiler can be disabled with the following flag to reduce profiling data used to decide whether to optimize or not a method.

-XX:+TieredCompilation -XX:TieredStopAtLevel=1

The code space is reduced by 20MB. Moreover, the memory outside JVM is reduced by 80MB (difference between NMT space and RSS space). The optimizing compiler C2 needs 100MB.

The C1 and C2 compilers can be disabled with the following flag.

-Xint

The memory outside the JVM is now lower than the total committed space. The code space is reduced by 43MB. Beware, this has a major impact on the performance of the application. Disabling C1 and C2 compiler reduces the memory used by 170 MB.

Using Graal VM compiler (replacement of C2) leads to a bit smaller memory footprint. It increases of 20MB the code memory space and decreases of 60MB from outside JVM memory.

The article Java Memory Management for JVM provides some relevant information the different memory spaces. Oracle provides some details in Native Memory Tracking documentation. More details about compilation level in advanced compilation policy and in disable C2 reduce code cache size by a factor 5. Some details on Why does a JVM report more committed memory than the Linux process resident set size? when both compilers are disabled.