Python pysftp get_r from Linux works fine on Linux but not on Windows

Question

I would like to copy an entire directory structure with files and subfolders recursively using SFTP from a Linux server to a local machine (both Windows and Linux)using Python 2.7.

I am able to ping the server and download the files using WinSCP from the same machine.

I tried the following code, works fine on Linux but not on Windows.

I tried \, /, os.join, all gives me same error, checked permissions as well.

import os
import pysftp

cnopts = pysftp.CnOpts()
cnopts.hostkeys = None    # disable host key checking.
sftp=pysftp.Connection('xxxx.xxx.com', username='xxx',password='xxx',cnopts=cnopts)
sftp.get_r('/abc/def/ghi/klm/mno', 'C:\pqr',preserve_mtime=False)

File "<stdin>", line 1, in <module> File "C:\Python27\lib\site-packages\pysftp_init_.py", line 311, in get_r preserve_mtime=preserve_mtime)
File "C:\Python27\lib\site-packages\pysftp_init_.py", line 249, in get self._sftp.get(remotepath, localpath, callback=callback)
File "C:\Python27\lib\site-packages\paramiko\sftp_client.py", line 769, in get with open(localpath, 'wb') as fl: IOError: [Errno 2] No such file or directory: u'C:\\pqr\\./abc/def/ghi/klm/mno/.nfs0000000615c569f500000004' 

Answer 1

Indeed, pysftp get_r does not work on Windows. It uses os.sep and os.path functions for remote SFTP paths, what is wrong, as SFTP paths always use a forward slash.

But you can easily implement a portable replacement:

import os
from stat import S_IMODE, S_ISDIR, S_ISREG

def get_r_portable(sftp, remotedir, localdir, preserve_mtime=False):
    for entry in sftp.listdir_attr(remotedir):
        remotepath = remotedir + "/" + entry.filename
        localpath = os.path.join(localdir, entry.filename)
        mode = entry.st_mode
        if S_ISDIR(mode):
            try:
                os.mkdir(localpath)
            except OSError:     
                pass
            get_r_portable(sftp, remotepath, localpath, preserve_mtime)
        elif S_ISREG(mode):
            sftp.get(remotepath, localpath, preserve_mtime=preserve_mtime)

Use it like:

get_r_portable(sftp, '/abc/def/ghi/klm/mno', 'C:\\pqr', preserve_mtime=False) 

---

Possible modifications of the code:

• Do not download empty folders while downloading from SFTP server using Python
• Download files from SFTP server that are older than 5 days using Python
• How to sync only the changed files from the remote directory using pysftp?

---

Side note: Do not "disable host key checking". You are losing a protection against MITM attacks.

For a correct solution, see Verify host key with pysftp.