Multiprocessing: use only the physical cores?

后端 未结 3 803
误落风尘
误落风尘 2021-01-30 11:25

I have a function foo which consumes a lot of memory and which I would like to run several instances of in parallel.

Suppose I have a CPU with 4 physical cor

3条回答
  •  醉酒成梦
    2021-01-30 11:53

    I found a solution that doesn't involve changing the source code of a python module. It uses the approach suggested here. One can check that only the physical cores are active after running that script by doing:

    lscpu
    

    in the bash returns:

    CPU(s):                8
    On-line CPU(s) list:   0,2,4,6
    Off-line CPU(s) list:  1,3,5,7
    Thread(s) per core:    1
    

    [One can run the script linked above from within python]. In any case, after running the script above, typing these commands in python:

    import multiprocessing
    multiprocessing.cpu_count()
    

    returns 4.

提交回复
热议问题