How can multithreading speed up an application (when threads can't run concurrently)?

Question

I\'m learning about multithreading, but after reading some tutorials I\'m sort of confused. I don\'t understand how multithreading can speed up an application.

By in

Answer 1

Two ways I can think of, the first of which is probably what you mean by "parallel threading".

• If you have multiple CPUs or cores, they can work simultaneously if you're running multiple threads.
• In the single core case, if your thread ends up waiting for (synchronous) I/O, let's say you call read() to read 100 MB from tape, another thread can get scheduled and get work done while you wait.

Answer 2

Removing 'parallel threading' from the concept of multithreading does make it pointless — if you don't allow the threads to execute at the same time then all you've got is one stream of processing that spends a lot of time hopping about in the OS's scheduler.

That the threads can operate in parallel is the whole performance gain. You should optimise your code so that semaphores are rarely used, if ever — you're right that they're expensive. A common approach is thread pooling and event loops; suppose you had 2,000 objects that you wanted to mutate, you'd push 2,000 associated tasks to the thread pool. The thread pool would ensure that the individual actions are performed on such threads as become available, when they become available. If it's then able to post an event into a defined event loop when the work is done, there are no explicit semaphores in your code at all.

Answer 3

because you constantly have to wait for those semaphores.

Only in a poorly-designed program or in one designed for parallel work on a single-processor machine. In a well-designed program, the threads do useful work in parallel in between the synchronization points, and enough of it to outweigh the overhead of synchronization.

Even without parallel (multicore/multiprocessor) processing, multithreading can be beneficial when the threads do blocking I/O. E.g., the good old CVSup programs used multithreading in the single-core era to make full use of network connections' duplex capabilities. While one thread was waiting for data to arrive over the link, another would be pushing data the other way. Due to network latency, both threads necessarily had to spend a lot of time waiting, during which the other threads could do useful work.

Answer 4

Not everything happens on CPU. Imagine a computer that doesn't have threads. That computer will waste extremely large amounts of time:

• waiting for keyboard to respond
• waiting for mouse to respond
• waiting for hard drive to complete request
• waiting for network packet to arrive from some destination

and so on. In fact, such a computer won't be able to do anything with the CPU, if a system is designed to be minimally interactive.

Same thing, at a lesser extent, applies to ONE process i.e. your application.

EDIT:

Before 'nice' kernels that run on 'nice' processors such as 286 and from there, OS-es (or primitive-OS-es) were simulating multithreading by handling interrupts. Even ZX Spectrum had interrupts to handle keyboard, for example (if I remember correctly).

Answer 5

On a computer, many programs (or threads) share some resources. Suppose one thread is waiting for a specific resource (for example, it wants to write data to disk). OS can then switch to another tread to be able to continue computing using available resources. This is why it is often a good idea to put I/O operations on a separate thread, and also to put GUI in a separate thread.

Off course, multithreading will not gives you a perfect speedup, but it can help a little big increasing performances. It is even better on hyperthreading architectures, where some registers are duplicated to minimize impact of context switching.

Answer 6

The idea behind multithreading is to have as few blocking points as possible. In other words, if a thread has to constantly wait on another thread to finish something, then the benefit of threads is likely lost in that situation.

Obligatory link: http://en.wikipedia.org/wiki/Amdahl's_law

Also, as Mark Ransom said, if your hardware can't actually do more than 1 thing at once, then threads are really just logically running at the same time (by swapping) than actually running at the same time. That can still be useful in situations with IO blocking though.