Why is Having More Threads than Cores Faster?

Question

I\'ve implemented a version of PageRank in a multithreaded version. I\'m running it on a 4-core Q6600. When I run it set to create 4 threads, I get:

real             


        

Answer 1

Deliberately creating more threads than processors is a standard technique used to make use of "spare cycles" where a thread is blocked waiting for something, whether that's I/O, a mutex, or something else by providing some other useful work for the processor to do.

If your threads are doing I/O then this is a strong contender for the speed-up: as each thread blocks waiting for the I/O, the processor can run the other threads until they too block for I/O, hopefully by which time the data for the first thread is ready, and so forth.

Another possible cause of the speed up is that your threads are experiencing false sharing. If you have two threads writing data to different values on the same cache line (e.g. adjacent elements of an array) then this will block the CPU whilst the cache line is transferred back and forth. By adding more threads you decrease the likelihood that they are operating on adjacent elements, and thus reduce the chance of false sharing. You can easily test this by adding extra padding to your data elements so they are each at least 64 bytes in size (the typical cache line size). If your 4-thread code speeds up, this was the problem.

Answer 2

You probably have spare CPU cycles while the thread blocks for some resources like memory. These CPU cycles can be used by other threads. The data I'd look at is... Does the 4 thread version show 100% utilization of each core? If not the you've found your spare CPU cycles.