Java process memory usage keeps increasing infinitely

Question

Preconditions:

• PC with 16 Gb of RAM
• JDK 1.8.x installed on Ubuntu 16.10 x64.
• a standard Spring-based web application, that is deployed on Tomcat

Answer 1

Can you rerun your tests with this option -XX:MaxDirectMemorySize=1024m? The exact value of this limit does not matter, but it shows possible "leaks".

Can you also provide GC details (-XX:+PrintGC)?

java.nio.ByteBuffer is a possible cause of them due to it specific finalization.

UPDATE #1

I have seen similar behavior for two another reasons: java.misc.Unsafe (unlikely) and high-loaded JNI-calls.

It is hard to understand without profile of the test.

UPDATE #2

Both high-loaded JNI-calls and finalize() method cause the described problem since objects do not have enough time to finalize.

The j.u.zip.Inflater fragment below:

/**
 * Closes the decompressor when garbage is collected.
 */
protected void finalize() {
    end();
}

/**
 * Closes the decompressor and discards any unprocessed input.
 * This method should be called when the decompressor is no longer
 * being used, but will also be called automatically by the finalize()
 * method. Once this method is called, the behavior of the Inflater
 * object is undefined.
 */
public void end() {
    synchronized (zsRef) {
        long addr = zsRef.address();
        zsRef.clear();
        if (addr != 0) {
            end(addr);
            buf = null;
        }
    }
}

private native static void end(long addr);

Answer 2

My hypothesis is that you collect allocation info / call stacks / etc in JProfiler and the RSS growth you observe is related to JProfiler keeping that data in memory.

You can verify if that's true by collecting less info (there should be a screen at the start of the profiling allowing you to e.g. not collect object allocations) and see if you observe smaller RSS growth as a result. Running your load test without JProfiler is also an option.

I had a similar case in the past.

Answer 3

Based on Occam's razor: couldn't it be that you have somewhere a memory leak (i.e., "unintentional object retentions" a'la Effective Java Item 6)?