Measure code speed in .net in milliseconds

Question

I want to get the maximum count I have to execute a loop for it to take x milliseconds to finish.

For eg.

int GetIterationsForExecutionTime(int ms)
{
            


        

Answer 1

I want to get the maximum count I have to execute a loop for it to take x milliseconds to finish.

First off, simply do not do that. If you need to wait a certain number of milliseconds do not busy-wait in a loop. Rather, start a timer and return. When the timer ticks, have it call a method that resumes where you left off. The Task.Delay method might be a good one to use; it takes care of the timer details for you.

If your question is actually about how to time the amount of time that some code takes then you need much more than simply a good timer. There is a lot of art and science to getting accurate timings.

First you should always use Stopwatch and never use DateTime.Now for these timings. Stopwatch is designed to be a high-precision timer for telling you how much time elapsed. DateTime.Now is a low-precision timer for telling you if it is time to watch Doctor Who yet. You wouldn't use a wall clock to time an Olympic race; you'd use the highest precision stopwatch you could get your hands on. So use the one provided for you.

Second, you need to remember that C# code is compiled Just In Time. The first time you go through a loop can therefore be hundreds or thousands of times more expensive than every subsequent time due to the cost of the jitter analyzing the code that the loop calls. If you are intending on measuring the "warm" cost of a loop then you need to run the loop once before you start timing it. If you are intending on measuring the average cost including the jit time then you need to decide how many times makes up a reasonable number of trials, so that the average works out correctly.

Third, you need to make sure that you are not wearing any lead weights when you are running. Never make performance measurements while debugging. It is astonishing the number of people who do this. If you are in the debugger then the runtime may be talking back and forth with the debugger to make sure that you are getting the debugging experience you want, and that chatter takes time. The jitter is generating worse code than it normally would, so that your debugging experience is more consistent. The garbage collector is collecting less aggressively. And so on. Always run your performance measurements outside the debugger, and with optimizations turned on.

Fourth, remember that virtual memory systems impose costs similar to those of jitters. If you are already running a managed program, or have recently run one, then the pages of the CLR that you need are likely "hot" -- already in RAM -- where they are fast. If not, then the pages might be cold, on disk, and need to be page faulted in. That can change timings enormously.

Fifth, remember that the jitter can make optimizations that you do not expect. If you try to time:

// Let's time addition!
for (int i = 0; i < 1000000; ++i) { int j = i + 1; }

the jitter is entirely within its rights to remove the entire loop. It can realize that the loop computes no value that is used anywhere else in the program and remove it entirely, giving it a time of zero. Does it do so? Maybe. Maybe not. That's up to the jitter. You should measure the performance of realistic code, where the values computed are actually used somehow; the jitter will then know that it cannot optimize them away.

Sixth, timings of tests which create lots of garbage can be thrown off by the garbage collector. Suppose you have two tests, one that makes a lot of garbage and one that makes a little bit. The cost of the collection of the garbage produced by the first test can be "charged" to the time taken to run the second test if by luck the first test manages to run without a collection but the second test triggers one. If your tests produce a lot of garbage then consider (1) is my test realistic to begin with? It doesn't make any sense to do a performance measurement of an unrealistic program because you cannot make good inferences to how your real program will behave. And (2) should I be charging the cost of garbage collection to the test that produced the garbage? If so, then make sure that you force a full collection before the timing of the test is done.

Seventh, you are running your code in a multithreaded, multiprocessor environment where threads can be switched at will, and where the thread quantum (the amount of time the operating system will give another thread until yours might get a chance to run again) is about 16 milliseconds. 16 milliseconds is about fifty million processor cycles. Coming up with accurate timings of sub-millisecond operations can be quite difficult if the thread switch happens within one of the several million processor cycles that you are trying to measure. Take that into consideration.

Answer 2

You could also use the Stopwatch class:

int GetIterationsForExecutionTime(int ms)
{
    int count = 0;
    Stopwatch stopwatch = new Stopwatch();
    stopwatch.Start();        
    do
    {
        // some code here
        count++;
    } while (stopwatch.ElapsedMilliseconds < ms);

    stopwatch.Stop();
    return count;
}

Answer 3

var sw = Stopwatch.StartNew();
...
long  elapsedMilliseconds = sw.ElapsedMilliseconds;

Answer 4

Good points from Eric Lippert. I'd been benchmarking and unit testing for a while and I'd advise you should discard every first-pass on you code cause JIT compilation. So in a benchmarking code which use loop and Stopwatch remember to put this at the end of the loop:

                // JIT optimization.
                if (i == 0)
                {
                    // Discard every result you've collected.
                    // And restart the timer.
                    stopwatch.Restart();
                }