0

I am trying to copy all the files from a directory (about 200GB of data) on an old USB drive (WD MyBook 3T) to a new USB drive (WD Element 2T). First I used drag & copy from with Windows 10. But Windows 10 skipped many files because permission. So I decided to use rsync under ubuntu. Since some of the files are copied over already, I thought it is going to be rather quick. So I did sudo -i, and cd to the mount directory of the old USB, and run rsync

rsync -avH --delete Backup/ ../EL2T2/Backup/

Lo and behold, 4 days and nights over, the rsync process is still running, running, and running, excruciatingly slooooooow.

The source directory has a lot of hard links, which might contributing to the slowness, but it is very hard to believe it can be this slow.

Any reason for the slowness? Anyway to solve this? Should I just kill rsync and run dd?

BTW, this is ubuntu 18.04.2, and both source and destination are NTFS.

9
  • 1
    You cannot get faster than rsync. The "-a" option is not appropriate for file systems that do not support linux permissions. Do hard links on ntfs drives work the same as on native linux file systems? – vanadium Apr 12 '19 at 15:29
  • Unlikely to be rsync's fault. The old USB connection is more likely to be the bottleneck. If some files were copied with Windows, you may have to play with rsync options to ignore some properties and avoid copying over again (check output to see if existing files are being copied again). You should be able to kill it and resume the process but be aware that depending on the file system you will have to use different options to ensure, as I said before, that rsync doesn't copy files that already exist in the destination. You shouldn't need --delete it if you are just backing up a folder. – Katu Apr 12 '19 at 15:31
  • It took only 37 seconds to dd a 4GB file from the old usb drive to the new usb drive. So I'm not sure the old usb is the bottleneck. It could be some kind issue between rsync and ntfs, I guess. – Guangliang Apr 12 '19 at 15:38
  • Have you tried -a --no-links ? Maybe the symbolic links are posing a problem. – 24601 Apr 12 '19 at 16:19
  • But I want to copy over the symbolic link information. – Guangliang Apr 12 '19 at 16:32
0

I also had a super slow rsync for some reason, then I built a backup server and had permission problems and it would quit copying due to only regular user privelages on the backup server, even though I have full sudo privelages on both machines and ran sudo rsync, on the backup server I didn't have the elevated privelage to root. I now run rsync as a daemon on my backup server which is 10 years old and sata 2, this allows me to also run as root on the backup server (destination machine) or as any user I choose, the first thing I noticed is how blazing fast it is to rsync my entire 8TB LVM volume. For an example a 6GB Movie MKV used to take around 20 minutes, Not 5 minutes ago I just synced up 4 movie files = 7.6GB, my timer gave me 1 minute and 48 Seconds. everything - permissions, ACLs', symlinks, hardlinks, etc. are exactly the same as far as I could tell, unfortunately I have had to use my backup to duplicate all data to another server and everything worked flawlessly.

1
  • Try giving some good formatting to your answer... Welcome to Ask Ubuntu. – ThunderBird May 10 '20 at 4:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.