Java Performance Processes vs Threads

Question

I am implementing a worker pool in Java.

This is essentially a whole load of objects which will pick up chunks of data, process the data and then store the result. Beca

Answer 1

Actually, if you do this with large scale taks using multiple jvm process is way faster than one jvm with multple threads. At least we never got one jvm runnning as fast as multple jvms.

We do some calculations where each task uses around 2-3GB ram and does some heavy number crunching. If we spawn 30 jvm's and run 30 task they perform around 15-20% better than spawning 30 threads in one jvm. We tried tuning the gc and the various memory sections and never catched up to the first variant.

We did this on various machines 14 tasks on a 16 core server, 34 tasks on a 36 core server etc. Multithreading in java always performed worde than multiple jvm processes.

It may not make any difference on simple tasks but on heavy calculations it seems jvm performce bad on threads.