Read remote file using python subprocess and ssh?

Question

How can I read data from a big remote file using subprocess and ssh?

Answer 1

For performance improvement, which is important when the file is big, there is rsync. For more information about the exact improvement see following post and the answer from Rafa: How does `scp` differ from `rsync`?

The algorithm would then be the following using rsync

import subprocess

subprocess.Popen(["rsync", host-ip+'/path/to/file'],stdout=subprocess.PIPE)
for line in ssh.stdout:
    line  # do stuff