Get JVM to grow memory demand as needed up to size of VM limit?

Question

We ship a Java application whose memory demand can vary quite a lot depending on the size of the data it is processing. If you don\'t set the max VM (virtual memory) size

Answer 1

I think the easiest way to do this would be to launch the JVM via some wrapper application, which would check system resources to determine memory availability, and then launch the JVM with the appropriate -Xmx parameter.

The question then becomes how that wrapper would be written. It may even be possible for the wrapper app to itself be a JVM, although I don't think the API or system properties would expose the necessary information. Maybe a shell script or your choice could get the information.

Answer 2

I do not think either the Sun or IBM JVM can do this (I know that the AS/400 one can, but that is most likely not relevant to you).

I would suggest using Java WebStart (and before you discard this, then notice that it has been updated with Java 6 u 10 and is much better suited for launching "local" applications and applet) since it allows you to provide a "small instance", "larger instance", "gigantic instance" as links/icons.

You will most likely look into the "inject application in webstart cache" and "offline"options.

Answer 3

I don't think you can do what you are trying to do; instead you'll have to ship instructions specific to your customers, their systems and their demands of how they can modify your .cmd file to allow for more memory.

Of course, if your product is aimed at very non-technical users, you may wish to hide this behind some more user-friendly config file. E.g.

# HIGH, MEDIUM, LOW - please change as appropriate. The guidelines are:
#                       * HIGH - users who generate 500 items per day
#                       * MEDIUM - 200-500 items etc
memoryUsage=MEDIUM

or possibly deploy different config files depending on which product option a user specifies when they order the product in the first place.

Answer 4

This discussion is moot if you think that your clients can ask for 2-3GB of RAM on their 32-bit machine. The OS and other apps will be taking their pound of flesh to run as well.

Sounds like your app is reaching the point where it needs a 64-bit operating system and lots more RAM.

Answer 5

One more option... I work on a launcher called WinRun4J, which allows you to specify a max heap size as a percentage of the available memory on the machine its running on (ie. it does a check for the amount of memory available and sets the -Xmx parameter dynamically on startup).

The INI option is "vm.heapsize.max.percent". There is also another option "vm.heapsize.preferred", which sets the -Xmx parameter as the maximum available memory on the machine up to this amount.

I believe some of the other launchers (eg. Launch4J, Janel) offer the same functionality.

Answer 6

In comments you say that the amount of memory that your application actually depends on the input dataset size provided by the user. This suggests that instead of trying to grab all available virtual memory (which may cause problems for the user's other applications) you should be looking at the input dataset size before you start the JVM and using that to estimate the amount of memory the application will need.

Suppose that the user's machine is configured with modest physical memory and a huge swap space. If you launch the JVM with a huge VM size, it could cause severe "thrashing" as the JVM tries to access data in non-resident pages. By contrast, if you give the JVM something more than the application needs and less than the available physical memory, you should be able to run comfortably without thrashing.