I have a Java process/app. When I run /usr/sbin/lsof -p on that java process, I see a lot of "can't identify protocol". Also, interestingly, File descriptors(FDs) are increasing at a very steady rate. And those FDs that are being created are having description as "can't identify protocol".
So, is there any way to instrument/profile the java process so as to nail down who is creating that many FDs. Any detailed explanation on any tool would be really really helpful.
A quick google search tells me strace is one way but IIUC, that will show linux system calls coming out of the java process. I am more interested in which part of my java code is behaving badly than what system calls are being generated.
Again, any ideas/suggestions would be simply great!
Lsof prints can't identify protocol for half-open TCP/IP connections: https://idea.popcount.org/2012-12-09-lsof-cant-identify-protocol/
When lsof prints "Can't identify protocol", this usually relates to sockets (it should also say 'sock' in the relevant output lines).
So, somewhere in your code you are probably connecting sockets and not closing them properly (perhaps you need a finally block).
I suggest you step through your code with a debugger (easiest to use your IDE, potentially with a remote debugger, if necesssary), while running lsof side-by-side. You should eventually be able to see which thread / line of code is creating these File Descriptors.
See point 10.2.2 of this FAQ for more details about the Lsof output.
Maybe you observe bug in JVM 1.6 with leak of connections when you use SSL and hostname: http://bugs.sun.com/view_bug.do?bug_id=6745052
