Can I auto restart tomcat jvm on out of memory exception

Question

I know that this is not \"best practice\" but I would like to know if I can auto restart tomcat if my deployed app throws an outofmemory exception

Answer 1

What about something like this? -XX:OnOutOfMemoryError="exec \`ps --no-heading -p $$ -o cmd\`"

Answer 2

Generally, no. The VM is a bad state, and cannot be completely trusted.

Typically, one can use a configurable wrapper process that starts and stops the "real" server VM you want. An example I've worked with is "Java Service Wrapper" from Tanuki Software http://wrapper.tanukisoftware.com/doc/english/download.jsp

I know there are others.

To guard against OOMs in the first place, there are ways to instrument modern VMs via interface beans to query the status of the heap and other memory structures. These can be used to, say, warn in a log or an email if some app specific operations are pushing some established limits.

Answer 3

Unfortunately when you kill the java process. Your script will keep a reference to the tomcat ports 8080 8005 8009 and you will not be able to start it again from the same script. The only way it works for me is:

-XX:OnOutOfMemoryError="kill -9 %p" and then another cron or monit or something similar to ensure you have the tomcat running again.

%p is actually the JVM pid , something the JVM provides for you.

Answer 4

I use

-XX:OnOutOfMemoryError='pkill java;/usr/local/tomcat/bin/start.sh'

Answer 5

I know this isn't what you asked, but have you tried looking through a heap dump to see where you may be leaking memory?

Some very useful tools for tracking down memory leaks:

jdk/bin/jmap -histo:live pid

This will give you a histogram of all live objects currently in the JVM. Look for any odd object counts. You'll have to know your application pretty well to be able to determine what object counts are odd.

jdk/bin/jmap -dump:live,file=heap.hprof pid

This will dump the entire heap of the JVM identified by pid. You can then use the great Eclipse Memory Analyzer to inspect it and find out who is holding on to references of your objects. Your two biggest friends in Eclipse Memory Analyzer are the histo gram and a right click -> references -> exclude weak/soft references to see what is referencing your object.

jconsole is of course another good tool.

Answer 6

You can try to use the OnOutOfMemoryError JVM option

-XX:OnOutOfMemoryError="/yourscripts/tomcat-restart"

It is also possible to generate the heap dump for later analysis:

-XX:+HeapDumpOnOutOfMemoryError

Be careful with combining these two options. If you force killing the process in "tomcat-restart" the heap dump might not be complete.