recursively use scp but excluding some folders

Question

Assume there are some folders with these structures

/bench1/1cpu/p_0/image/
/bench1/1cpu/p_0/fl_1/
/bench1/1cpu/p_0/fl_1/
/bench1/1cpu/p_0/fl_1/
/bench1/1cpu/p_0         


        

Answer 1

Although scp supports recursive directory copying with the -r option, it does not support filtering of the files. There are several ways to accomplish your task, but I would probably rely on find, xargs, tar, and ssh instead of scp.

find . -type d -wholename '*bench*/image' \
| xargs tar cf - \
| ssh user@remote tar xf - -C /my/dir

The rsync solution can be made to work, but you are missing some arguments. rsync also needs the r switch to recurse into subdirectories. Also, if you want the same security of scp, you need to do the transfer under ssh. Something like:

rsync -avr -e "ssh -l user" --exclude 'fl_*' ./bench* remote:/my/dir