Why using parallel streams in static initializer leads to not stable deadlock

Question

CAUTION: it is not a duplicate, please read topic сarefully https://stackoverflow.com/users/3448419/apangin quote:

T

Answer 1

TL;DR This is a HotSpot bug JDK-8215634

The problem can be reproduced with a simple test case that has no races at all:

public class StaticInit {

    static void staticTarget() {
        System.out.println("Called from " + Thread.currentThread().getName());
    }

    static {
        Runnable r = new Runnable() {
            public void run() {
                staticTarget();
            }
        };

        r.run();

        Thread thread2 = new Thread(r, "Thread-2");
        thread2.start();
        try { thread2.join(); } catch (Exception ignore) {}

        System.out.println("Initialization complete");
    }

    public static void main(String[] args) {
    }
}

This looks like a classic initialization deadlock, but HotSpot JVM does not hang. Instead it prints:

Called from main
Called from Thread-2
Initialization complete

Why this is a bug

JVMS §6.5 requires that upon execution of invokestatic bytecode

the class or interface that declared the resolved method is initialized if that class or interface has not already been initialized

When Thread-2 calls staticTarget, the main class StaticInit is obviously uninitialized (since its static initializer is still running). This means Thread-2 must launch class initialization procedure described in JVMS §5.5. According to this procedure,

2. If the Class object for C indicates that initialization is in progress for C by some other thread, then release LC and block the current thread until informed that the in-progress initialization has completed

However, Thread-2 is not blocked despite the class is in progress of initialization by thread main.

What about other JVMs

I tested OpenJ9 and JET, and they both expectedly deadlock on the above test.
It's interesting that HotSpot also hangs in -Xcomp mode, but not in -Xint or mixed modes.

How it happens

When interpreter first encounters invokestatic bytecode, it calls JVM runtime to resolve the method reference. As a part of this process JVM initializes the class if necessary. After successful resolution the resolved method is saved in the Constant Pool Cache entry. Constant Pool Cache is a HotSpot-specific structure that stores resolved constant pool values.

In the above test invokestatic bytecode that calls staticTarget is first resolved by the main thread. Interpreter runtime skips class initialization, because the class is already being initialized by the same thread. The resolved method is saved in the constant pool cache. The next time when Thread-2 executes the same invokestatic, the interpreter sees that the bytecode is already resolved and uses constant pool cache entry without calling to runtime and thus skips class initialization.

A similar bug for getstatic/putstatic was fixed long ago - JDK-4493560, but the fix did not touch invokestatic. I've submitted the new bug JDK-8215634 to address this issue.

As to the original example,

whether it hangs or not depends on which thread first resolves the static call. If it is main thread, the program completes without a deadlock. If the static call is resolved by one of ForkJoinPool threads, the program hangs.

Update

The bug is confirmed. It is fixed in the upcoming releases: JDK 8u201, JDK 11.0.2 and JDK 12.