Why does park/unpark have 60% CPU usage?

Question

Recently we started stress testing our application (an XMPP based chat server) using YJP 11.0.9. During our test we noticed following strange behavior.

1. Sampling sh

Answer 1

High CPU time of Unsafe.unpark is a sign of excessive Context Switching. Here is an article to get the idea of how expensive is a context switch:

How long does it take to make a context switch?

The easiest option to find out CS count on Linux is run vmstat .

Once high CS is confirmed (e.g. more 10K switches per core/per second) you take the offending thread (you can sort threads in YJP by their CPU time) and run strace -p -c to find out the cause of switching, e.g. thread is reading from socket in small pieces and get's switched off, in which case increasing socket buffer might help.