Measuring time in Java

Question

So I was trying to measure the time two different algorithm implementations took to accomplish a given task, and here is the result:

i    alg1  alg2
4   0.00         


        

Answer 1

You should consider running your algorithms many times and averaging the result.

There are three reasons for this:

1. It makes timing easier, for the reasons you have identified.

2. The JVM "warms up" and the performance of your code will change as it does: The hotspot JVM won't compile fully until you've run a method a large number of times. If you don't get past this your results won't be representative.

3. It's always a good idea to average out the times, to avoid spurious effects due to external events like GC, or other stuff running on your computer.

As a rule of thumb, try running your algos 10,000 times as a warm up, then 10,000 times afterwards. Modify the numbers to suit your runtimes...

Here's an article explaining the same thing.

Answer 2

You could instrument the code and count the number of bytecodes executed. You can do this with bycounter.

This may not be ideal for your purposes. If the programs differ by only a very few bytecodes it may not give an accurate measure of which program is actually more performant, as the cost of executing bytecodes can vary wildly. Also, if there is network or disk reads during your program, the bytecode count could give the wrong comparison.

Answer 3

For basic timing, you can use Guava's Stopwatch class (or just grab its source code if you don't want to pull in the whole Guava library). For a more complete benchmarking solution look at Caliper by the same team.

Both of these are based on System.nanoTime(), which you should prefer over System.currentTimeMillis() for measuring elapsed time. The basic reason why is that System.currentTimeMillis() is a "clock" (which tries to return wall-time) whereas System.nanoTime() is a "timer" (which tries to return time since some arbitrary point).

You want a clock when you're trying to figure out when a single event happened, so you can line it up with your watch or the clock on your wall (or the clock in some other computer). But it's not appropriate for measuring the elapsed time between two events on the same system, since the computer will occasionally adjust its notion of how its own internal clock corresponds to wall-time. For instance, if you do

long timeA = System.currentTimeMillis();
doStuff();
long timeB = System.currentTimeMillis();
System.out.println("Elapsed time: " + (timeB - timeA));

it's possible to get a negative result if NTP adjusts backwards while doStuff() is executing. System.nanoTime(), being a timer instead of a clock, should ignore that adjustment thus avoid this problem.

(Note that all of the above is conceptual; unfortunately things can get messy at the implementation level. But this doesn't change the recommendation: System.nanoTime() is supposed to be the best timer you can get on your platform, and System.currentMilliseconds() is supposed to be the best clock you can get.)

Answer 4

System.nanoTime(); perhaps, is what you're looking for. And add calculations for standard deviation and average time.