When I measure the throughput of my Java application, I see a 50% performance increase over time:
Are you sure it is CPU related and not IO related? I have seen this behaviour a lot of times, when hitting a cold cache somewhere worsens performance.
You could try adapting your app to run it with Nailgun. Instead of invoking your app against a fresh JVM each time you invoke it against a Nailgun server which is a long-lived JVM. The second time you invoke your app, the nailgun JVM will have optimized the paths in your classes and should therefore execute a lot faster than it would from fresh.
Use '-server' to do much more up front. Hotspot does not as far as I know allow for saving jit information between runs so -server is the simplest way to tell it what you want it to do.
Few more options to tune up the JIT.
1. Class Data sharing http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/index.jsp?topic=%2Fcom.ibm.java.doc.user.aix64.60%2Fuser%2Fclassdatasharing.html
2. Tiered Compilation See details for flag -XX:+TieredCompilation
3. Custom CompileThreshold Controls number of invocations of a function that would make it eligible for JIT compile. See details for flag -XX:CompileThreshold. Never make this to ZERO or ONE. Your tampering here may result in deterioration of performance. JVM gives you the option though. -server defaults this to 10000.