How to filter files when using scp to copy dir recursively?

Question

I need to copy all the .class files from server to local with all dir reserved. e.g. server:/usr/some/unknown/number/of/sub/folders/me.class will be /usr/proj

Answer 1

If you indeed wanna use scp, there's a indirect way.Say we want to copy all .jpg file under local folder '/src' to folder '/dst' in remote server 10.1.1.2:

#make a clean temp folder
mkdir /tmp/ttt
#copy all .jpg file and retain folder structure as-is
find /src -type f -name *.jpg -exec cp --parents \{\} /tmp/ttt \;
#copy to remote target folder as-is and retain original time attributes
scp -rp /tmp/ttt/* 10.1.1.2:/dst
#if copy ok, remove temp folder
rm -rf /tmp/ttt

Answer 2

Since you can scp you should be ok to ssh,
either script the following or login and execute...

# After reaching the server of interest
cd /usr/some/unknown/number/of/sub/folders
tar cfj pack.tar.bz2 $(find . -type f -name *.class)

return back (logout) to local server and scp,

# from the local machine
cd /usr/project/backup/some/unknown/number/of/sub/folders
scp you@server:/usr/some/unknown/number/of/sub/folders/pack.tar.bz2 .
tar xfj pack.tar.bz2

---

If you find the $(find ...) is too long for your tar change to,

find . -type f -name *.class | xargs tar cfj pack.tar.bz2

---

Finally, since you are keeping it in /usr/project/backup/,
why bother extraction? Just keep the tar.bz2, with maybe a date+time stamp.

Answer 3

With ssh key based authentication enabled, the following script would work.

for x in `ssh user@remotehost 'find /usr/some -type f -name *.class'`; do y=$(echo $x|sed 's/.[^/]*$//'|sed "s/^\/usr//"); mkdir -p /usr/project/backup$y; scp $(echo 'user@remotehost:'$x) /usr/project/backup$y/; done