rsync : Recursively sync all files while ignoring the directory structure

Question

I am trying to create a bash script for syncing music from my desktop to a mobile device. The desktop is the source.

Is there a way to make rsync recursively sync files

Answer 1

The answer to your question: No, rsync cannot do this alone. But with some help of other tools, we can get there... After a few tries I came up with this:

rsync -d --delete $(find . -type d|while read d ; do echo $d/ ; done) /targetDirectory && rmdir /targetDirectory/* 2>&-

The difficulty is this: To enable deletion of files at the target position, you need to:

1. specify directories as sources for rsync (it doesn't delete if the source is a list of files).
2. give it the complete list of sources at once (rsync within a loop will give you the contents of the last directory only at the target).
3. end the directory names with a slash (otherwise it creates the directories at the target directory)

So the command substitution (the stuff enclosed with the $( )) does this: It finds all directories and adds a slash (/) at the end of the directory names. Now rsync sees a list of source directories, all terminated with a slash and so copies their contents to the target directory. The option -d tells it, not to copy recursively.

The second trick is the rmdir /targetDirectory/* which removes the empty directories which rsync created (although we didn't ask it to do that).

I tested that here, and deletion of files removed in the source tree worked just fine.

Answer 2

May be this is a recent option, but I see the option --no-relative mentioned in the documentation for --files-from and it worked great.

find SourceDir -name \*.mp3 | rsync -av --files-from - --no-relative . DestinationDir/

Answer 3

If you can make a list of files, you've already solved the problem. Try:

find /path/to/src/ -name \*.mp3 > list.txt
rsync -avi --no-relative --progress --files-from=list.txt / user@server:/path/to/dest

If you run the script again for new files, it will only copy the missing files.

If you don't like the list, then try a single sentence (but it's another logic)

find /path/to/src/ -name \*.mp3 -type f \
  -exec rsync -avi --progress {} user@server:/path/to/dest/  \;

In this case, you will ask for each file, each time, since by the type of sentence, you cannot build the file list previously.

Answer 4

Simply:

rsync -a --delete --include=*.mp3 --exclude=* \
    pathToSongs/Theme*/Artist*/. destuser@desthost:Music/.

would do the job if you're path hierarchy has a fixed number of level.

WARNING: if two song file do have exactly same name, while on same destination directory, your backup will miss one of them!

If else, and for answering strictly to your ask ignoring the directory structure you could use bash's shopt -s globstar feature:

shopt -s globstar
rsync -a --delete --include=*.mp3 --exclude=* \
    pathToSongsRoot/**/. destuser@desthost:Music/.

At all, there is no need to fork to find command.

Recursively sync all files while ignoring the directory structure

For answering strictly to question, there must no be limited to an extension:

shopt -s globstar
rsync -d --delete sourceRoot/**/. destuser@desthost:destRoot/.

With this, directories will be copied too, but without content. All files and directories would be stored on same level at destRoot/.

WARNING: If some different files with same name exists in defferents directories, they would simply be overwrited on destination, durring rsync, for finaly storing randomly only one.