Java program is getting slower after running for a while

Question

I have a java program that is a typical machine learning algorithm, updating the values for some parameters by some equations:

for (int iter=0; iter<1000; ite         


        

Answer 1

Giving Java (or any garbage collecting language) too much memory has an adverse effect on performance. The live (referenced) objects become increasing sparse in memory resulting in more frequent fetches from main memory. Note that in the examples you've shown us the faster windows is doing more quick and full GC than Linux - but GC cycles (especially full gcs) are usually bad for performance.

If running the training set does not take an exceptionally long time, then try benchmarking at different memory allocations.

A more radical solution, but one which should have a big impact is to eliminate (or reduce as much as possible) object creation within the loop by recycling objects in pools.

Answer 2

If the GC time is hundreds of milliseconds as shown in your screenshot then GC is likely not the issue here. I suggest you look into lock contention and possibly IO using a profiler (Netbeans is great). I know you stated your program did very little IO but with profiling (much like debugging) you have to remove all your assumptions and go step by step.

Answer 3

I'm sorry to post this as an answer but I don't have enough score to comment.

If you think it's a GC related issue I'd change it for the Garbage 1 Collector –XX:+UseG1GC

I found this brief explanation about it: http://blog.takipi.com/garbage-collectors-serial-vs-parallel-vs-cms-vs-the-g1-and-whats-new-in-java-8/

Can you run your software under profiling? Try to use the jprofiler, VisualVM or even the netbeans profiler. It may help you a lot.

I noticed that you have your own encapsulation of a vector and matrix. Maybe your are spending a lot more memory than necessary with that too. But I don't think that is the problem.

Sorry again about not contributing as a comment. (It would be more appropriate)

Answer 4

First, it is a common best practice to declare Variables outside of loops to avoid garbace collection. as 'Wagner Tsuchiya' said, try running a profiler if you have doubts about the GC. If you want some tips on GC tuning, i found nice blogpost.

Answer 5

You could try calling System.gc() every couple iterations to see if performance goes up or down. This may help you narrow it down to some of the previous answers diagnostics.

Answer 6

I would consider declaring the vars outside the loop so mem allocation is done once and eliminate GC completely.