I have a function foo
which consumes a lot of memory and which I would like to run several instances of in parallel.
Suppose I have a CPU with 4 physical cor
I found a solution that doesn't involve changing the source code of a python module. It uses the approach suggested here. One can check that only the physical cores are active after running that script by doing:
lscpu
in the bash returns:
CPU(s): 8
On-line CPU(s) list: 0,2,4,6
Off-line CPU(s) list: 1,3,5,7
Thread(s) per core: 1
[One can run the script linked above from within python]. In any case, after running the script above, typing these commands in python:
import multiprocessing
multiprocessing.cpu_count()
returns 4.