How to avoid [Errno 12] Cannot allocate memory errors caused by using subprocess module

前端 未结 3 1328
忘掉有多难
忘掉有多难 2020-12-04 15:32

Complete Working Test Case

Of course depending on your memory on the local and remote machines your array sizes will be different.

         


        
相关标签:
3条回答
  • 2020-12-04 16:12

    If you are running out of memory, you may want to increase your swap memory. Or you might have no swap enabled at all. In Ubuntu (it should work for other distributions as well) you can check your swap by:

    $sudo swapon -s
    

    if it is empty it means you don't have any swap enabled. To add a 1GB swap:

    $sudo dd if=/dev/zero of=/swapfile bs=1024 count=1024k
    $sudo mkswap /swapfile
    $sudo swapon /swapfile
    

    Add the following line to the fstab to make the swap permanent.

    $sudo vim /etc/fstab
    
         /swapfile       none    swap    sw      0       0 
    

    Source and more information can be found here.

    0 讨论(0)
  • 2020-12-04 16:13

    If you're running out of memory, it's probably because subprocess is trying to read too much into memory at the same time. The solution, other than using a redirect to a local file, is probably to use popen-like functionality with an stdin/stdout pair that can be read from a little at a time.

    0 讨论(0)
  • 2020-12-04 16:27

    This should do it:

    http://docs.python.org/3.3/library/subprocess.html#replacing-os-popen-os-popen2-os-popen3

    That way, you can read lines or blocks, instead of the entire thing at once.

    0 讨论(0)
提交回复
热议问题