2

Today I was using rsync over a slow internet connection to transfer a video file - uncompressed avi, 60 seconds duration, 360MB in size. Basically it contained 1500 frames of the same image, hence it was highly compressible.

Top upload speed on my internet connection is 130KB/s. I ran rsync like this:

rsync -avhz --progress --partial myfile.avi remote:/path/

The transfer speed varied from 400KB/s to 1,5MB/s. I accidentially Ctrl+C'd the running rsync after the file was about halfway transferred. But no worries, I used --partial switch, so I just restarted rsync with the same parameters as before. Only that now the transfer speed was 80MB/s!

I tried this again, and again and again. Even without the -z switch, the result was the same. On the first run, the transfer speed was a shy 1,5MB/s, but after interrupting rsync after a while and restarting it, the rest of the file was transferred almost instantaneously. The md5sum matched on both ends.

My question is: can anyone explain this behaviour to me? Am I missing something obvious here? I would expect that file would be transferred much faster already with -z switch on the first go, but I wouldn't expect it to speed up so drastically without even using -z after resuming the transfer.

  • -h, --help show this help screen is a typo? – Anders F. U. Kiær Nov 15 '13 at 2:56
  • Nope, -h, --human-readable output numbers in a human-readable format – Brane Nov 15 '13 at 10:08
2

man rsync

From description:

"It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination."

So the speed you got indicated more or less how quick rsync could confirm that the file was already transferred or confirm and transfer the changed/missing parts.

  • I guess it just bothered me that if I would run rsync only once, it would take for example 15 minutes to transfer that file. But if I would interrupt the transfer after a few seconds and restart it, it would transfer the rest of the file in seconds, which is a huge difference. Is there any way in a case like this to tell rsync to use it's delta-transfer algorithm between already transferred chunks of a file and original file without having to interrupt it? – Brane Nov 15 '13 at 2:36
  • As far as I know does rsync always use it. Did you delete\move the file on the destination? Before trying to start it fresh as you described and cancel and restart it? Eg. if you transfer 1/3 of the file in the first go. And then resume the file, will it only be 2/3 left to transfer but rsync will show the speed of transferring the complete file, as it do compare the whole file. – Anders F. U. Kiær Nov 15 '13 at 2:47
  • I deleted the file on the destination before each rsync+break+rsync attempt.I attempted to transfer the whole file in one go and it took about 15 minutes. When I transferred the file by starting rsync for 10 seconds, Ctrl+C-ing it, and restarting rsync, the file transfer completed in the next 5 seconds. – Brane Nov 15 '13 at 3:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.