recursively use scp but excluding some folders

Question

Assume there are some folders with these structures

/bench1/1cpu/p_0/image/
/bench1/1cpu/p_0/fl_1/
/bench1/1cpu/p_0/fl_1/
/bench1/1cpu/p_0/fl_1/
/bench1/1cpu/p_0         


        

Answer 1

If you use a pem file to authenticate u can use the following command (which will exclude files with something extension):

rsync -Lavz -e "ssh -i  -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --exclude "*.something" --progress  @:

The -L means follow links (copy files not links). Use full path to your pem file and not relative.

Using sshfs is not recommended since it works slowly. Also, the combination of find and scp that was presented above is also a bad idea since it will open a ssh session per file which is too expensive.