One of my applications hangs under some period of running under load, does anyone know what could cause such output in jstack:
\"scheduler-5\" prio=10 tid=0x
unsafe.park is pretty much the same as thread.wait, except that it's using architecture specific code (thus the reason it's 'unsafe'). unsafe is not made available publicly, but is used within java internal libraries where architecture specific code would offer significant optimization benefits. It's used a lot for thread pooling.
So, to answer your question, all the thread is doing is waiting for something, it's not really using any CPU. Considering that your original stack trace shows that you're using a lock I would assume a deadlock is going on in your case.
Yes I know you have almost certainly already solved this issue by now. However, you're one of the top results if someone googles sun.misc.unsafe.park. I figure answering the question may help others trying to understand what this method that seems to be using all their CPU is.
I had a similar issue, and following previous answers (thanks!), I was able to search and find how to handle correctly the ThreadPoolExecutor terminaison.
In my case, that just fix my progressive increase of similar blocked threads:
ExecutorService::awaitTermination(x, TimeUnit)
and ExecutorService::shutdownNow()
(if necessary) in my finally clause.For information, I've used the following commands to detect thread count & list locked threads:
ps -u javaAppuser -L|wc -l
jcmd `ps -C java -o pid=` Thread.print >> threadPrintDayA.log
jcmd `ps -C java -o pid=` Thread.print >> threadPrintDayAPlusOne.log
cat threadPrint*.log |grep "pool-"|wc -l
From the stack trace it's clear that, the ThreadPoolExecutor > Worker thread started and it's waiting for the task to be available on the BlockingQueue(DelayedWorkQueue) to pick the task and execute.So this thread will be in WAIT status only as long as get a SIGNAL from the publisher thread.