Is HyperThreading / SMT a flawed concept?

Question

The primary idea behind HT/SMT was that when one thread stalls, another thread on the same core can co-opt the rest of that core\'s idle time and run with it, transparently.

Answer 1

1. Not all programmers have enough knowledge, time and many other things to write efficient, cache-friendly programs. Most of the time only the critical parts are optimized when needed. The other parts may have lots of cache misses
2. Even if the program was written with cache efficiency in mind, it may not eliminate cache misses completely. Cache availability is a dynamic information only known at runtime, and neither the programmer nor the compiler knows that to optimize memory access.
   • Cache unpredictability is one of the reasons the Itanium failed, because while the compilers can reorder arithmetic operations, it cannot guess those cache information in a multithreading environment to reorder memory loads/stores efficiently.
   • Each time the cache misses, hundreds of cycles are wasted which can be useful for other purposes. Some CPUs do out-of-order execution. But even OoO execution has its limits and you'll be blocked at some point. During those time while waiting for all memory problems to be solved, you can switch to another CPU thread and continue running.
3. It's not only Intel that uses SMT now. AMD Bulldozer has module multithreading which is a partial SMT. There are still lots of other architectures that use SMT such as MIPS, PowerPC... There are even CPUs with 8 or 16 threads per core, like with 12-core 96-thread power8 CPUs

https://en.wikipedia.org/wiki/Simultaneous_multithreading#Modern_commercial_implementations

Update:

AMD has moved to full SMT now in the Zen microarchitecture