Best way to test code speed in C++ without profiler, or does it not make sense to try?

Question

On SO, there are quite a few questions about performance profiling, but I don\'t seem to find the whole picture. There are quite a few issues involved and most Q & A ignore

Answer 1

Is it safe to say you're asking two questions?

• Which one is faster, and by how much?

• And why is it faster?

For the first, you don't need high precision timers. All you need to do is run them "long enough" and measure with low precision timers. (I'm old-fashioned, my wristwatch has a stop-watch function, and it is entirely good enough.)

For the second, surely you can run the code under a debugger and single-step it at the instruction level. Since the basic operations are so simple, you will be able to easily see roughly how many instructions are required for the basic cycle.

Think simple. Performance is not a hard subject. Usually, people are trying to find problems, for which this is a simple approach.

Answer 2

(This answer is specific to Windows XP and the 32-bit VC++ compiler.)

The easiest thing for timing little bits of code is the time-stamp counter of the CPU. This is a 64-bit value, a count of the number of CPU cycles run so far, which is about as fine a resolution as you're going to get. The actual numbers you get aren't especially useful as they stand, but if you average out several runs of various competing approaches then you can compare them that way. The results are a bit noisy, but still valid for comparison purposes.

To read the time-stamp counter, use code like the following:

LARGE_INTEGER tsc;
__asm {
    cpuid
    rdtsc
    mov tsc.LowPart,eax
    mov tsc.HighPart,edx
}

(The cpuid instruction is there to ensure that there aren't any incomplete instructions waiting to complete.)

There are four things worth noting about this approach.

Firstly, because of the inline assembly language, it won't work as-is on MS's x64 compiler. (You'll have to create a .ASM file with a function in it. An exercise for the reader; I don't know the details.)

Secondly, to avoid problems with cycle counters not being in sync across different cores/threads/what have you, you may find it necessary to set your process's affinity so that it only runs on one specific execution unit. (Then again... you may not.)

Thirdly, you'll definitely want to check the generated assembly language to ensure that the compiler is generating roughly the code you expect. Watch out for bits of code being removed, functions being inlined, that sort of thing.

Finally, the results are rather noisy. The cycle counters count cycles spent on everything, including waiting for caches, time spent on running other processes, time spent in the OS itself, etc. Unfortunately, it's not possible (under Windows, at least) to time just your process. So, I suggest running the code under test a lot of times (several tens of thousands) and working out the average. This isn't very cunning, but it seems to have produced useful results for me at any rate.