1

Yet another slow transfer problem. I have a newly bought SanDisk USB 3.0 drive. Laptop only has USB 2.0, but I am getting transfer speeds of ~2MB/s consistently.

I am transferring a .img to the drive:

dd if=my.img of=/dev/sdb status=progress

Both the above and iostat show a consistent 2MB/s.

kern.log when connecting gives:

usb 2-1.2: new high-speed USB device number 6 using ehci-pci
usb 2-1.2: New USB device found, idVendor-0781, idProduct=5583
usb 2-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 2-1.2: Product: Ultra Fit
usb 2-1.2: Manufacturer: SanDisk
usb 2-1.2: SerialNumber: 4C53....
usb-storage 2-1.2:1.0: USB Mass Storage device detected
scsi host7: usb-storage 2-1.2:1.0
scsi 7:0:0:0 Direct-Access SanDisk Ultra Fit 1.00 PQ: 0 ANSI: 6
sd 7:0:0:0 Attached scsi generic sg2 type 0
sd 7:0:0:0 [sdb] 30031250 512-byte logical blocks: (15.4 GB/14.3GiB)
sd 7:0:0:0 [sdb] Write Protect is off
sd 7:0:0:0 [sdb] Mode Sense: 43 00 00 00
sd 7:0:0:0 [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
sd 7:0:0:0 [sdb] Attached SCSI removable disk

Relevant part of lsusb -t output:

/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver-ehci-pci/3p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/8p, 480M
        |__ Port 2: Dev 6, If 0, Class=Mass Storage, Driver=usb-storage, 480M

Running Ubuntu 16.04 LTS Server n Dell 6320.

To add drivers:

# lsmod | grep pci
pci_stub       16384 1
vboxpci        24576 0
vboxdrv       454656 3 vboxnetadp,vboxnetflt,vboxpci
sdhci_pci      28672 0
sdhci          45056 1 sdhci_pci

# lsmod | grep hci
ahci           36864 4
sdhci_pci      28672 0
hibachi        32768 1
sdhci          45056 1 sdhci_pci
4

You should simply increase the block size of dd. The default is 1 single byte, but you can drastically speed the process up with transferring larger data chunks at once. The responsible parameter is called bs.

You might want to experiment a bit with the optimal values, but something around 4kB (4096 bytes) should be okay. Some sources also recommend larger block sizes of e.g. 32kB, 64kB or even 128kB. It depends strongly on the target drive's physical structure which value is optimal and can almost only be found out by experiment. You can think of 4kB as lower boundary though, smaller block sizes are usually much slower.

More information about optimal block sizes for dd can be found in this related question on Unix & Linux.

dd if=my.img of=/dev/sdb bs=4096 status=progress
  • Beat me by 15 seconds :) – Android Dev May 9 '16 at 11:33
  • Really? That simple? I was concerned with screwing up the transferred filesystem, but I guess with a dd img copy it doesn't matter, right? I will try it now. – deitch May 9 '16 at 11:33
  • @deitch Yes, that simple. But 4kB is still a pretty small value. You might maybe rather want to try something bigger like 32kB or 64kB. It's strongly depending on the target drive, so you should experiment and try transferring some smaller files and timing those until you find a good value. – Byte Commander May 9 '16 at 11:37
  • Definitely a dramatic improvement. It started off at several hundred MB/s and deteriorated by 15MB/s by the time 2GB had transferred. Any idea why it deteriorates? Still a massive improvement! – deitch May 9 '16 at 11:43
  • 1
    When transferring data over USB, the system uses a write cache in RAM. Depending on your RAM size and load, it may have a varying capacity. So when you send data to write over to the USB device, it actually gets buffered in the said cache first, and then the system reads it from there and writes it to the device in background until it's done. However, the program (here dd) does not notice this caching and assumes that all data the OS has accepted from it has been written to the disk, even though it is only in the cache yet. Therefore until the cache is full, you see ultra high transfer rates – Byte Commander May 9 '16 at 11:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.