should I “bind” “spinning” thread to the certain core?

Question

My application contains several latency-critical threads that \"spin\", i.e. never blocks. Such thread expected to take 100% of one CPU core. However it seems modern operation s

Answer 1

Running a thread locked to a single core gives the best latency for that thread in most circumstances if this is the most important thing in your code.

The reasons(R) are

• your code is likely to be in your iCache
• the branch predictors are tuned to your code
• your data is likely to be ready in your dCache
• the TLB points to your code and data.

Unless

• Your running a SMT sytem (ex. hyperthreaded) in which case the evil twin will "help" you with by causing your code to be washed out, your branch predictors to be tuned to its code and its data will push your out of the dCache, your TLB is impacted by its use.
  • Cost unknown, each cache misses cost ~4ns, ~15ns and ~75ns for data, this quickly runs up to several 1000ns.
  • It saves for each reason R mentioned above, that is still there.
  • If the evil twin also just spins the costs should be much lower.
• Or your allowing interrupts on your core, in which case you get the same problems and
  • your TLB is flushed
  • you take a 1000ns-20000ns hit on the context switch, most should be in the low end if the drivers are well programmed.
• Or you allow the OS to switch your process out, in which case you have the same problems as the interrupt, just in the hight end of the range.
  • switching out could also cause the thread to pause for the entire slice as it can only be run on one (or two) hardware threads.
• Or you use any system calls that cause context switches.
  • No disk IO at all.
  • only async IO else.
• having more active (none-paused) threads than cores increases the likelihood of problems.

So if you need less than 100ns latency to keep your application from exploding you need to prevent or lessen the impact of SMT, interrupts and task switching on your core. The perfect solution would be an Real time operating system with static scheduling. This is a nearly perfect match for your target, but its a new world if your have mostly done server and desktop programming.

The disadvantages of locking a thread to a single core are:

• It will cost some total throughput.
  • as some threads that might have run if the context could have been switched.
  • but the latency is more important in this case.
• If the thread gets context switched out it will take some time before it can be scheduled potentially one or more time slices, typically 10-16ms, which is unacceptable in this application.
  • Locking it to a core and its SMT will lessen this problem, but not eliminate it. Each added core will lessen the problem.
  • setting its priority higher will lessen the problem, but not eliminate it.
  • schedule with SCHED_FIFO and highest priority will prevent most context switches, interrupts can still cause temporary switches as does some system calls.
  • If you got a multi cpu setup you might be able to take exclusive ownership of one of the CPU's through cpuset. This prevents other applications from using it.

Using pthread_setschedparam with SCHED_FIFO and highest priority running in SU and locking it to the core and its evil twin should secure the best latency of all of these, only a real time operating system can eliminate all context switches.

Other links:

Discussion on interrupts.

Your Linux might accept that you call sched_setscheduler, using SCHED_FIFO, but this demands you got your own PID not just a TID or that your threads are cooperative multitasking.
This might not ideal as all your threads would only be switches "voluntarily" and thereby removing flexibility for the kernel to schedule it.

Interprocess communication in 100ns