Concurrency and Multithreading

Question

I\'m not very experienced with subjects such as Concurrency and Multithreading. In fact, in most of my web-development career I had never needed to touch these subjects.

Answer 1

The language choice depends on the application you want to do.

Do you want to create a highly scalable system, with lots of incoming "requests"? Then Erlang may be a good choice. It is known to be a good fit for "highly concurrent" application scenarios.

If you want to write a typical game and want it to use the now typically available dual or quad core CPUs of your audience, then you are bound to different decisions (frameworks, engines, libraries, available hardware interfaces). In this case, you will use threads, and thread pools to offload processing work. Most probably you will use a kind of message queue to communicate between the threads.

In (server-side) web-development you most probably already gained some experience in concurrent programming! Maybe you were not aware of it, because the language and the given framework (maybe Apache & PHP) provided you with an environment that took the burden off your shoulders.

Answer 2

I'm studying the subject (right now :D) and one of the things that seems a very important difference on concurrency between languages is the language expressive power on concurrency.

For example C++ has no native support to concurrency and it rely on functions provided by the OS.

Java is a step above because has some built-in methods while others are left to the OS (threads scheduling or priority for example).

Instead one of what seems to be one of the best programming languages supporting concurrency is Ada which has in fact a whole concurrency-model built in (scheduling and priority included).

Why is this important? Because of the portability!

Using a language with good concurrency expressive power allows you to bring your concurrent program to Windows, linux or Mac withouth great fears about the way it will work. For example: thread priority will be applied in the same way in your Ada program running in windows, linux or Mac while it can be really different (ignored in some OS and applied in others) with Java or C++.

This is what seem to me by the course I'm taking at the university right now :)

Answer 3

Well for one thing, multiple threads are not the same as multiple processes, so fork() really does not apply here.

Multithreading/parallel processing is hard. First you have to figure out how to actually partition the task to be done. Then you have to coordinate all of the parallel bits, which may need to talk to each other or share resources. Then you need to consolidate the results, which in some cases can be every bit as difficult as the previous two steps. I'm simplifying here, but hopefully you get the idea.

So your question is, why would some languages be better at it? Well, several things can make it easier:

• Optimized immutable data structures. You want to stick to immutable structures whenever possible in parallel processing, because they are much easier to reason about. Some languages have better support for these, and some have various optimizations, i.e. the ability to splice collections together without any actual copying while still enforcing the immutability. You can always build your own structures like these, but it's easier if the language or framework does it for you.

• Synchronization primitives and ease of using them. When different threads do share state, they need to be synchronized and there are many different ways to accomplish this. The wider the array of sync primitives you get, the easier your task will ultimately be. Performance will take a hit if you have to sync with a critical section instead of a reader-writer lock.

• Atomic transactions. Even better than a wide array of sync primitives is not having to use them at all. Database engines are very good at this; instead of you, the programmer, having to figure out exactly which resources you need to lock and when and how, you just say to the compiler or interpreter, "all of the stuff below this line needs to happen together, so make sure nobody else messes around with it while I'm using it." And the engine will figure out the locking for you. You almost never get this kind of simplicity in an abstract programming language, but the closer you can come, the better. Thread-safe objects that combine multiple common operations into one are a start.

• Automatic parallelism. Let's say you have to iterate through a long list of items and transform them somehow, like multiply 50,000 10x10 matrices. Wouldn't it be nice if you could just tell the compiler: Hey, each operation can be done independently, so use a separate CPU core for each one? Without having to actually implement the threading yourself? Some languages support this kind of thing; for example, the .NET team has been working on PLINQ.

Those are just a few examples of things that can make your life easier in parallel/multi-threaded applications. I'm sure that there are many more.

Answer 4

To fork is human to thread is divine :D

Forking involves the kernel and creates a seperate adress space -- meaning proc X and Y can't easily share and need to use IPC primitives, and creating a thread allows in process synchronization which is FAR faster than IPC which involves kernel context switches by both processes and needless thread yield's (which involve the kernel to wake said thread up).

However there is a multitude of reasons why different concurrency models are better than others. I'm just giving you the rough rule of thumb for general programming. For example not forking might endanger logic which could be separated out by forking (I love that word) -- endangered because if the other logic in the process crashes it , well said logic is going down with the process.

Answer 5

Concurrency is basically being able to fork() processes and compute stuff in parallel the same way memory management is basically being able to call malloc. It is part of the story, but not all of it. Being able to simplify the issues relating to concurrency is the difference between languages that are good at concurrency and those that just can be concurrent.

Answer 6

In languages that are not designed for concurrency, you must rely upon low-level system calls and manage a lot of things yourself. In contrast, a programming language designed for concurrency, like Erlang, will provide high-level constructs that hide the low-level details. This makes it easier to reason about the correctness of your code, and also results in more portable code.

Also, in a programming language designed for concurrency, there are typically only a handful of ways to do concurrent things, which leads to consistency. In contrast, if the programming language was not designed for concurrency, then different libraries and different programmers will do things in different ways, making it difficult to make choices about how to do them.

It's a bit like the difference between a programming language with automated garbage collection and one without. Without the automation, the programmer has to think a lot about implementation details.

The difference between multithreaded programming and multi-process programming (i.e., fork()), is that a multithreaded program may be more efficient because data doesn't have to be passed across process boundaries, but a multi-process approach may be more robust.