1

I'm backing around 2Tb of data from a ntfs disk to another ntfs disk with rsync ( I tried also with midnight commander ) and the copy starts with a "good" 25Mb/s speed, but after copying a couple of gigabytes the speed drops down to around 5Mb/s sometimes even less.

If I stop the copy and start again rsync to continue the copy the speed starts again around 25Mb/s and then goes back to 5Mb/s.

This is the start of rsync, from here everything copies that slowly.

>f+++++++++ _University Courses/Harvard/Harvard - Advanced Algorithms 2016/11-Advanced Algorithms (COMPSCI 224), Lecture 11.mp4
    502,527,183 100%   25.29MB/s    0:00:18 (xfr#2, ir-chk=1021/53136)
>f+++++++++ _University Courses/Harvard/Harvard - Advanced Algorithms 2016/12-Advanced Algorithms (COMPSCI 224), Lecture 12.mp4
    494,046,164 100%   25.45MB/s    0:00:18 (xfr#3, ir-chk=1020/53136)
>f+++++++++ _University Courses/Harvard/Harvard - Advanced Algorithms 2016/13-Advanced Algorithms (COMPSCI 224), Lecture 13.mp4
    389,502,911 100%   25.77MB/s    0:00:14 (xfr#4, ir-chk=1019/53136)
>f+++++++++ _University Courses/Harvard/Harvard - Advanced Algorithms 2016/14-Advanced Algorithms (COMPSCI 224), Lecture 15.mp4
    401,384,534 100%   14.92MB/s    0:00:25 (xfr#5, ir-chk=1018/53136)
>f+++++++++ _University Courses/Harvard/Harvard - Advanced Algorithms 2016/15-Advanced Algorithms (COMPSCI 224), Lecture 16.mp4
    498,564,894 100%    4.94MB/s    0:01:36 (xfr#6, ir-chk=1017/53136)
>f+++++++++ _University Courses/Harvard/Harvard - Advanced Algorithms 2016/16-Advanced Algorithms (COMPSCI 224), Lecture 17.mp4
    417,205,204 100%    2.30MB/s    0:02:52 (xfr#7, ir-chk=1016/53136)
>f+++++++++ _University Courses/Harvard/Harvard - Advanced Algorithms 2016/17-Advanced Algorithms (COMPSCI 224), Lecture 18.mp4
    495,885,960 100%    6.16MB/s    0:01:16 (xfr#8, ir-chk=1015/53136)
>f+++++++++ _University Courses/Harvard/Harvard - Advanced Algorithms 2016/18-Advanced Algorithms (COMPSCI 224), Lecture 19.mp4
    475,335,986 100%    2.75MB/s    0:02:45 (xfr#9, ir-chk=1014/53136)
>f+++++++++ _University Courses/Harvard/Harvard - Advanced Algorithms 2016/19-Advanced Algorithms (COMPSCI 224), Lecture 20.mp4
    485,359,371 100%    1.40MB/s    0:05:29 (xfr#10, ir-chk=1013/53136)
>f+++++++++ _University Courses/Harvard/Harvard - Advanced Algorithms 2016/20-Advanced Algorithms (COMPSCI 224), Lecture 21.mp4
    505,021,448 100%    6.46MB/s    0:01:14 (xfr#11, ir-chk=1012/53136)
>f+++++++++ _University Courses/Harvard/Harvard - Advanced Algorithms 2016/21-Advanced Algorithms (COMPSCI 224), Lecture 22.mp4

The drives are not noticeable fragmented.

I tried mounting the drives with the big_writes and the async option but I'd seen no difference.

I tested the speed with hdparm with and without cache and the speed is faster than those 25Mb/s.

I monitored with iostat and the %rwqm is very high, but I have no idea if that's bad or ok.

enter image description here

I'm trying to find the bottleneck or the problem but I haven't been able to locate it, any help on how could I monitor and analyze the problem would be appreciated.

Update: I tried disabling the disk write cache to check if the speed drop was happening after the drive filled the cache:

sudo hdparm -W0 /dev/sde

But the problem persist and the drive behaves the same.

Update:

$ free -h                                                                                                                          
total        used        free      shared  buff/cache   available                                                                  
Mem:          7.5Gi       1.1Gi       146Mi        13Mi       6.2Gi       6.1Gi                                                                  
Swap:            0B          0B          0B    
                                                                                              
$ sudo dmidecode -s bios-version                                                                                     
4.6.4                                                                                                                                            

$ sysctl vm.swappiness                                                                                               
  vm.swappiness = 60

$ top

top - 21:16:09 up 6 days, 23:40,  5 users,  load average: 3.45, 2.89, 2.58
Tasks: 179 total,   1 running, 178 sleeping,   0 stopped,   0 zombie
%Cpu(s): 12.5 us, 20.1 sy,  0.0 ni, 17.4 id, 48.4 wa,  0.0 hi,  1.6 si,  0.0 st
MiB Mem :   7689.6 total,    254.0 free,   1151.4 used,   6284.2 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.   6225.8 avail Mem 
                                                                    
PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND

7320 usuario   20   0  284924 172348   1288 S  13.5   2.2   1:53.63 rsync                                                                        
7305 usuario   20   0  174100  92956   2748 S  12.9   1.2   2:13.81 rsync                                                                       
17809 usuario   20   0 1754596 476276  18104 S  11.9   6.0   2080:14 kodi-x11                                                                    
27195 root      20   0   18972   9736   1840 S   7.9   0.1  58:03.15 mount.ntfs                                                                  
14377 root      20   0   19948  10536   1764 S   6.3   0.1 126:50.65 mount.ntfs                                                                  
13856 root      20   0       0      0      0 S   3.3   0.0  52:25.26 usb-storage                                                                   
132 root      20   0       0      0      0 S   3.0   0.0 221:29.02 kswapd0                                                                     
25821 usuario   20   0   23816  14980   2884 S   2.3   0.2   9:46.40 tmux: server                                                                 
4909 root      20   0       0      0      0 D

Update after swapfile created:

free -h pre slow down:

$ free -h
              total        used        free      shared  buff/cache   available
Mem:          7.5Gi       427Mi       135Mi        11Mi       7.0Gi       6.8Gi
Swap:         4.0Gi       3.0Mi       4.0Gi

free -h during slow down:

$ free -h
              total        used        free      shared  buff/cache   available
Mem:          7.5Gi       421Mi       127Mi        11Mi       7.0Gi       6.8Gi
Swap:         4.0Gi       8.0Mi       4.0Gi

But still low speed after a few files copied:

sending incremental file list
.d..t...... _University Courses/MIT/MIT - Computers System Security 2014/
>f+++++++++ _University Courses/MIT/MIT - Computers System Security 2014/19-20. Mobile Phone Security.mp4
    866,880,572 100%   28.03MB/s    0:00:29 (xfr#1, ir-chk=1003/53245)
>f+++++++++ _University Courses/MIT/MIT - Computers System Security 2014/20-21. Data Tracking.mp4
    666,687,288 100%   27.70MB/s    0:00:22 (xfr#2, ir-chk=1002/53245)
>f+++++++++ _University Courses/MIT/MIT - Computers System Security 2014/21-22. Guest Lecture by MIT IS&T.mp4
    716,033,570 100%   25.97MB/s    0:00:26 (xfr#3, ir-chk=1001/53245)
>f+++++++++ _University Courses/MIT/MIT - Computers System Security 2014/22-23. Security Economics.mp4
    693,628,642 100%   25.63MB/s    0:00:25 (xfr#4, ir-chk=1000/53245)
>f+++++++++ _University Courses/MIT/MIT - Design and Analysis of Algorithms 2015/01-1. Course Overview, Interval Scheduling.mp4
  1,069,906,667 100%   24.40MB/s    0:00:41 (xfr#5, ir-chk=1122/53368)
>f+++++++++ _University Courses/MIT/MIT - Design and Analysis of Algorithms 2015/02-2. Divide & Conquer - Convex Hull, Median Finding.mp4
    919,036,317 100%    3.29MB/s    0:04:26 (xfr#6, ir-chk=1121/53368)
>f+++++++++ _University Courses/MIT/MIT - Design and Analysis of Algorithms 2015/03-R1. Matrix Multiplication and the Master Theorem.mp4
     67,469,312   6%    4.71MB/s    0:03:23
5
  • 1
    search this site for "slow copy" and you will see you are not alone. The basic problem is write are slower than reads, system buffers fill up, and things grind to a crawl. Try the nocache rsync.... to see if that helps.
    – ubfan1
    Sep 13, 2020 at 17:53
  • 1
    Edit your question and show me free -h and top and sysctl vm.swappiness, and sudo dmidecode -s bios-version and tell me the EXACT make/model # of your computer or motherboard.
    – heynnema
    Sep 13, 2020 at 18:30
  • @ubfan1, I tried nocache, still the same problem...
    – Fabman
    Sep 13, 2020 at 19:23
  • @heynnema, I added the info. The computer is a small barebone, AMD E350 cpu and 8gb.
    – Fabman
    Sep 13, 2020 at 19:26
  • You've got no SWAP!
    – heynnema
    Sep 13, 2020 at 19:28

1 Answer 1

1

You've got no SWAP!

Let's create a 4G /swapfile...

Note: Incorrect use of the dd command can cause data loss. Suggest copy/paste.

sudo swapoff -a           # turn off swap
sudo rm -i /swapfile      # remove old /swapfile

sudo dd if=/dev/zero of=/swapfile bs=1M count=4096

sudo chmod 600 /swapfile  # set proper file protections
sudo mkswap /swapfile     # init /swapfile
sudo swapon /swapfile     # turn on swap
free -h                   # confirm 8G RAM and 4G swap

Confirm this line in /etc/fstab... and confirm no other “swap” lines...

/swapfile    none    swap    sw      0   0

reboot                    # reboot and verify operation

Update #1:

Creating a /swapfile did not fix the copy speed problem.

However, we laid down a fresh GPT partition table on the external drive, and created a fresh ext4 partition (native to Linux), instead of NTFS (not native to Linux), and copies are occurring at the expected speed.

Further testing to see if going back to a NTFS partition causes the problem again, proves that ntfs-3g mounted partitions are the problem. Suggest trying ntfs mounted partitions.

Update #2:

Solved!. It was the ntfs driver, ntfs and ntfs-3g are very very slow. I'd been looking around and I found that Paragon developed a free version of their commercial ntfs kernel space driver. I installed it and after ten hours of copying is still copying at 40Mb/s without hiccup. See https://www.paragon-software.com/home/ntfs-linux-professional/.

28
  • Created swapfile and restarting now. free -h informs of 4gb swap. I'll check speeds now and I'll come back with info.
    – Fabman
    Sep 13, 2020 at 19:36
  • @Fabman Are you using Files (Navigator) to copy files? Can you open another terminal window and monitor free -h during when the slow downs occur? Show it to me then please.
    – heynnema
    Sep 13, 2020 at 20:15
  • I'm using rsync on a tmux ssh session, no graphical interface. I added the results of the free -h command during the slow down.
    – Fabman
    Sep 13, 2020 at 20:21
  • I also tried with midnight commander from the same sessions with the same results
    – Fabman
    Sep 13, 2020 at 20:29
  • @Fabman Are you going over ethernet, or wifi? What, if anything, is between you and the target disks?
    – heynnema
    Sep 13, 2020 at 20:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .