Java Thread hanging without deadlocks

Question

I am currently having some trouble with a process that is hanging.

First of all, the below thread dump shows no deadlock. Another interesting thing is that the process is hanging at java.lang.System.identityHashCode method. The currently CPU usage of that process is 0%, sometimes peaks at 1%... normally the process uses 100%. It is using 1.7GB of RAM (which is normal for this process). The machine has plenty of CPUs, disk space and RAM available.

Attaching to process ID 4146, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 23.7-b01
Deadlock Detection:

No deadlocks found.

Thread 12761: (state = BLOCKED)


Thread 4159: (state = BLOCKED)


Thread 4158: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=135 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove() @bci=2, line=151 (Interpreted frame)
 - java.lang.ref.Finalizer$FinalizerThread.run() @bci=3, line=177 (Interpreted frame)


Thread 4157: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=503 (Interpreted frame)
 - java.lang.ref.Reference$ReferenceHandler.run() @bci=46, line=133 (Interpreted frame)


Thread 4147: (state = IN_VM)
 - java.lang.System.identityHashCode(java.lang.Object) @bci=0 (Compiled frame; information may be imprecise)
 - java.util.IdentityHashMap.hash(java.lang.Object, int) @bci=1, line=301 (Compiled frame)
 - java.util.IdentityHashMap.containsKey(java.lang.Object) @bci=17, line=358 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=18, line=346 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.FastClonerArrayList.clone(java.lang.Object, com.rits.cloning.Cloner, java.util.Map) @bci=47, line=23 (Compiled frame)
 - com.rits.cloning.Cloner.fastClone(java.lang.Object, java.util.Map) @bci=30, line=96 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=100, line=355 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.FastClonerArrayList.clone(java.lang.Object, com.rits.cloning.Cloner, java.util.Map) @bci=47, line=23 (Compiled frame)
 - com.rits.cloning.Cloner.fastClone(java.lang.Object, java.util.Map) @bci=30, line=96 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=100, line=355 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.FastClonerArrayList.clone(java.lang.Object, com.rits.cloning.Cloner, java.util.Map) @bci=47, line=23 (Compiled frame)
 - com.rits.cloning.Cloner.fastClone(java.lang.Object, java.util.Map) @bci=30, line=96 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=100, line=355 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.FastClonerArrayList.clone(java.lang.Object, com.rits.cloning.Cloner, java.util.Map) @bci=47, line=23 (Compiled frame)
 - com.rits.cloning.Cloner.fastClone(java.lang.Object, java.util.Map) @bci=30, line=96 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=100, line=355 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.FastClonerArrayList.clone(java.lang.Object, com.rits.cloning.Cloner, java.util.Map) @bci=47, line=23 (Compiled frame)
 - com.rits.cloning.Cloner.fastClone(java.lang.Object, java.util.Map) @bci=30, line=96 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=100, line=355 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.cloneInternal(java.lang.Object, java.util.Map) @bci=368, line=404 (Compiled frame)
 - com.rits.cloning.Cloner.deepClone(java.lang.Object) @bci=61, line=309 (Compiled frame)
 - arquitetura.representation.Architecture.deepClone() @bci=23, line=497 (Compiled frame)
 - jmetal.operators.mutation.PLAFeatureMutation.execute(java.lang.Object) @bci=109, line=862 (Compiled frame)
 - jmetal.metaheuristics.nsgaII.NSGAII.execute() @bci=321, line=129 (Compiled frame)
 - br.ufpr.inf.opla.patterns.main.NSGAII_OPLA.main(java.lang.String[]) @bci=1116, line=179 (Interpreted frame)

I am executing the same process with different arguments for quite sometime now (more than a week). I am using evolutionary algorithms, so the time consuming is normal for my problem. I executed like 120 times the same process (with different arguments) and this one is the first one to hang in that time. There are currently 4 processes running simultaneously (counting the hanged one), some even more complexes that take a lot more time. They are finishing and my program is starting other processes normally using ProcessBuilder. I did read the output of each one. The hanged process outputted a normal log at less than halfway to its finish. No errors. Another important thing is that the method in which the process is frozen at generates no output, so I assume that there is no problem at the output consumption.

My questions are: I don't want to kill the process, so is there any way to unhang the process? Also, why is it at IN_VM state (Running in VirtualMachine) and is still frozen? Is it normal for a process to freeze at java.lang.System.identityHashCode method? Why is it using 0% (peaking 1% rarely) of my CPU?

If I can help with any other information, please just ask and I will be glad to post it here.

Thanks in advance.

EDIT: Although I use multithreading, each thread starts a Process object using ProcessBuilder. So, the hanging process is a separated JVM instance with only one main thread being executed. I redirect the output to a file, but as mentioned before, the process is hanged at a method that generates no output, so I assume that there is no problem with output consumption.

EDIT2: Killed the process. The subsequent process was started and is running smoothly.

Answer 1

The process is not hanging, you just simply happen to have found it in the state posted above. You can tell because the thread in question does not say BLOCKED as the others, so it is in normal execution.

Why the process is no longer executing might have different reasons, but I can assure you that none of them is associated with identityHashCode, as this is a well-tested internal Java function.

To proceed:

• Compare the JVM state of several hangups. If it is always the same as above, the issue might be somewhere associated with hash generation. If not:

• Post more relevant code so we can have a look at it. If the issue is not at this line of code, it has to be somewhere else (com.rits.cloning.Cloner sounds like a reasonable point to start looking).

• Tell us what you are trying to archive, someone might find a bug or a better way to do it, that completely circumvents whatever is causing the issue.