Multithreading: What is the point of more threads than cores?

Question

I thought the point of a multi-core computer is that it could run multiple threads simultaneously. In that case, if you have a quad-core machine, what\'s the point of having

Answer 1

I know this is a super old question with plenty of good answers, but I am here to point out something that is important in current environment:

If you want to design an application for multi-threading, you should not be designing for a specific hardware setting. CPU technology has been advancing quite rapidly for years, and core counts are increasing steadily. If you deliberately design your application such that it uses only 4 threads, then you are potentially restricting yourself in a octa-core system (for example). Now, even 20-core systems are commercially available, so such a design is definitely doing more harm than good.

Answer 2

Although you can certainly use threads for speeding up calculations depending on your hardware, one of their main uses is to do more than one thing at a time for user-friendliness reasons.

For example, if you have to do some processing in the background and also remain responsive to UI input, you can use threads. Without threads, the user interface would hang every time you tried to do any heavy processing.

Also see this related question: Practical uses for threads

Answer 3

Imagine a Web server that has to serve an arbitrary number of requests. You have to serve the requests in parallel because otherwise each new request has to wait until all the other requests have been completed (including sending the response over the Internet). In this case, most web servers have way less cores than the number of requests they usually serve.

It also makes it easier for the developer of the server: You only have to write a thread program that serves a request, you don't have to think about storing multiple requests, the order you serve them, and so on.

Answer 4

Threads can help with responsiveness in UI applications. Additionally, you can use threads to get more work out of your cores. For instance, on a single core, you can have one thread doing IO and another doing some computation. If it were single threaded, the core could essentially be idle waiting for the IO to complete. That's a pretty high level example, but threads can definitely be used to pound your cpu a bit harder.

Answer 5

Most of the answers above talk about performance and simultaneous operation. I'm going to approach this from a different angle.

Let's take the case of, say, a simplistic terminal emulation program. You have to do the following things:

• watch for incoming characters from the remote system and display them
• watch for stuff coming from the keyboard and send them to the remote system

(Real terminal emulators do more, including potentially echoing the stuff you type onto the display as well, but we'll pass over that for now.)

Now the loop for reading from the remote is simple, as per the following pseudocode:

while get-character-from-remote:
    print-to-screen character

The loop for monitoring the keyboard and sending is also simple:

while get-character-from-keyboard:
    send-to-remote character

The problem, though, is that you have to do this simultaneously. The code now has to look more like this if you don't have threading:

loop:
    check-for-remote-character
    if remote-character-is-ready:
        print-to-screen character
    check-for-keyboard-entry
    if keyboard-is-ready:
        send-to-remote character

The logic, even in this deliberately simplified example that doesn't take into account real-world complexity of communications, is quite obfuscated. With threading, however, even on a single core, the two pseudocode loops can exist independently without interlacing their logic. Since both threads will be mostly I/O-bound, they don't put a heavy load on the CPU, even though they are, strictly speaking, more wasteful of CPU resources than the integrated loop would be.

Now of course real-world usage is more complicated than the above. But the complexity of the integrated loop goes up exponentially as you add more concerns to the application. The logic gets ever more fragmented and you have to start using techniques like state machines, coroutines, et al to get things manageable. Manageable, but not readable. Threading keeps the code more readable.

So why would you not use threading?

Well, if your tasks are CPU-bound instead of I/O-bound, threading actually slows your system down. Performance will suffer. A lot, in many cases. ("Thrashing" is a common problem if you drop too many CPU-bound threads. You wind up spending more time changing the active threads than you do running the contents of the threads themselves.) Also, one of the reasons the logic above is so simple is that I've very deliberately chosen a simplistic (and unrealistic) example. If you wanted to echo what was typed to the screen then you've got a new world of hurt as you introduce locking of shared resources. With only one shared resource this isn't so much a problem, but it does start to become a bigger and bigger problem as you have more resources to share.

So in the end, threading is about many things. For example, it's about making I/O-bound processes more responsive (even if less efficient overall) as some have already said. It's also about making logic easier to follow (but only if you minimize shared state). It's about a lot of stuff, and you have to decide if its advantages outweigh its disadvantages on a case by case basis.

Answer 6

Just because a thread exists doesn't always mean it's actively running. Many applications of threads involve some of the threads going to sleep until it's time for them to do something - for instance, user input triggering threads to wake up, do some processing, and go back to sleep.

Essentially, threads are individual tasks that can operate independently of one another, with no need to be aware of the progress of another task. It's quite possible to have more of these than you have ability to run simultaneously; they're still useful for convenience even if they sometimes have to wait in line behind one another.