rsync suddenly hanging indefinitely during transfers

Question

For the past few years, I have been using an rsync one-liner to back up important folders on my Mac Mini desktop (OSX 10.9, 2.5 GHz i5, 4 GB RAM) to a FreeNAS box (0.7.2 Sabanda

Answer 1

I've seen this quite often on 3.0.9 on a directory with hardlinks, but it also happened on 3.1.3.

There is a nice analysis in Debian bug 820916: when its internal sockets are congested with errors, rsync could go into a deadlock.

This might have been fixed in a 3.2 release just a few days ago (Jun 2020):

Avoid a hang when an overabundance of messages clogs up all the I/O buffers.

The only good workaround I can think of is, if the problem is not persistent, then put timeout in front of it: timeout rsync <args> <source> <destination>, then retry. If it is persistent for you, you're the lucky one who can debug it :D

Answer 2

I've been running into the same thing again and again and it seems to help if you drop the -v option (which is annoying if you need that output).

Answer 3

I was having the same problem and it was because I was running out of memory during the rsync. Created a swap file and problem solved.

Answer 4

It also happens when the user on target machine has not write permissions on target folder.
You can try giving write permission to others target folder:

sudo chmod -R o+w /path/to/target-folder

Answer 5

I am using openSUSE 13.2 Linux, rsync version 3.1.1-2.4.1.x86_64, and I experienced similar problems, doing an rsync between my laptop and an external hard disk, with the destination device definitively having enough free space.

I thought I got an improvement omitting option -v, but after 10 minutes it was hanging again: strace said: select(5, [], [4], [], {60, 0}) = 0 (Timeout)

And with "iotop" I counld see confirm that the rsync processes did no significant disk IO any more.

Neither removing the -v option nor limiting the bandwidth using --bwlimit fixed the problem.

Answer 6

Most likely not "your" problem, but I stumbled upon this question when I was researching a similar behavior:

I'm observing "hanging" when the target site has too much io load. e.G. on one of my small business servers, when someone is resyncing his IMAP account and downloading large batchs of data and a backup job runs that writes his data.

In this situation I notice a steep drop in performance for rsync. Noticeable in a high load value in top on the target machine, even though CPU and Mem are fine.

Waiting for the process to finish has helped every time or interrupting and attempting the rsync at a later time again.