Complete Working Test Case
Of course depending on your memory on the local and remote machines your array sizes will be different.
If you are running out of memory, you may want to increase your swap memory. Or you might have no swap enabled at all. In Ubuntu (it should work for other distributions as well) you can check your swap by:
$sudo swapon -s
if it is empty it means you don't have any swap enabled. To add a 1GB swap:
$sudo dd if=/dev/zero of=/swapfile bs=1024 count=1024k
$sudo mkswap /swapfile
$sudo swapon /swapfile
Add the following line to the fstab
to make the swap permanent.
$sudo vim /etc/fstab
/swapfile none swap sw 0 0
Source and more information can be found here.
If you're running out of memory, it's probably because subprocess is trying to read too much into memory at the same time. The solution, other than using a redirect to a local file, is probably to use popen-like functionality with an stdin/stdout pair that can be read from a little at a time.
This should do it:
http://docs.python.org/3.3/library/subprocess.html#replacing-os-popen-os-popen2-os-popen3
That way, you can read lines or blocks, instead of the entire thing at once.