Multiprocessing inside a child thread

Question

I was learning about multi-processing and multi-threading.

From what I understand, threads run on the same core, so I was wondering if I create multiple processes insid

Answer 1

Every program is represented through one process. A process is the execution context one or multiple threads operate in. All threads in one process share the same tranche of virtual memory assigned to the process.

Python (refering to CPython, e.g. Jython and IronPython have no GIL) is special because it has the global interpreter lock (GIL), which prevents threaded python code from being run on multiple cores in parallel. Only code releasing the GIL can operate truely parallel (I/O operations and some C-extensions like numpy). That's why you will have to use the multiprocessing module for cpu-bound python-code you need to run in parallel. Processes startet with the multiprocessing module then will run it's own python interpreter instance so you can process code truely parallel.

Note that even a single threaded python-application can run on different cores, not in parallel but sequentially, in case the OS re-schedules execution to another core after a context switch took place.

Back to your question:

if I create multiple processes inside a child thread will they be limited to that single core too?

You don't create processes inside a thread, you spawn new independent python-processes with the same limitations of the original python process and on which cores threads of the new processes will execute is up to the OS (...as long you don't manipulate the core-affinity of a process, but let's not go there).

Answer 2

I'm not a pyhton expert but I expect this is like in other languages, because it's an OS feature in general.

Process

A process is executed by the OS and owns one thread which will be executed. This is in general your programm. You can start more threads inside your process to do some heavy calculations or whatever you have to do. But they belong to the process.

Thread

One or more threads are owned by a process and execution will be distributed across all cores.

Now to your question

When you create a given number of threads these threads should in general be distributed across all your cores. They're not limited to the core who's executing the phyton interpreter. Even when you create a subprocess from your phyton code the process can and should run on other cores.

You can read more about the gernal concept here: Preemptive multitasking

There're some libraries in different languages who abstract a thread to something often called a Task or something else. For these special cases it's possible that they're just running inside the thread they were created in. For example. In the DotNet world there's a Thread and a Task. Often people are misusing the term thread when they're talking about a Task, which in general runns inside the thread it was created.