I have an 8-disk Network Attached Storage device. I've been having issues with it where the hard drives fail on a regular basis. I figured out what the problem is with this: firmware issues with Seagate HDDs being compatible with my QNAP. So I need to update the firmware on each of the 8 HDDs. Each of the HDDs are 3TB in size. Previously, when one failed, I told the NAS to rebuild it (since I used RAID 5), and it took somewhere between 42-44 hours to do.

Current Problem:
I am using DD to capture an image of one of the 3TB HDDs. It is plugged into my other laptop via eSATA and the drive I am writing to is on a USB 3.0 connection. When I first started the image, I did the math to figure out how much time would be needed to finish. My calculations based on how long DD was running (via PS) and how much data had been written (via LS), it looked to be the same (42-44 hours). However, this process has been going on for almost a week. At the rate it ACTUALLY goes (realtime) it will take an entire week (7d to the hour exactly) for this to finish.

Right now, we're at 6d of DD running its course. When I type ps -ef | fgrep DISK1 I get the following:

root 6444 6388 36 Jan12 pts/0 2-04:54:14 dd if=/dev/sde of=/media/4TB/DISK1.raw

PS claims that this has only been running for 2d 4h, when in reality, it's been running for over 6d.

My questions:
Why is this? How can I fix it? Could it be a multi-threading issue (or lack of multi-threading I mean)? I did find that Ubuntu was suspending the OS when I closed the lid of the laptop, so I changed that very early on in the process so now it doesn't sleep, but it's still taking "forever".

  • Try specifying block size: bs=32M . The default is 512 bytes. Try up and down. It depends on the hardware.. – user.dz Jan 18 '14 at 19:06
  • 1
    I believe ps reports the CPU time consumed by dd, not the actual time it has run. As @Sneetsher mentioned, your dd command does not seem to have specified the block size. For most modern disks, 512 bytes is far too low to be optimal. You can also send a USR1 signal to the running dd process to find out the progress: kill -USR1 6444. – yjwong Jan 18 '14 at 19:15
  • That's what I figured, @yjwong :( – Damian T. Jan 21 '14 at 0:15
  • 2
    So, an update: it finished and I tried to fix my NAS and everything I tried didn't work, so now I have to create another image. I have downloaded PV to use with DD and I am trying the bs=32M that @Sneetsher suggested. So far, it is saying it will take around 26 hours, which is faster than it took for the NAS to rebuild a RAID disk. Kudos! – Damian T. Jan 21 '14 at 0:16
  • How did it pass? – user.dz Jan 25 '14 at 0:08

I found this QA while researching options for my own disappointing Seagate HDDs. You've got pv, which operates like cat with additional bonus of progress reporting. cat is faster than dd. Even with optimal block size in dd, cat is 30MB/sec+ faster.

I have pv /dev/sdd > /dev/sdc running and its speed varies from 170MB/sec to 195MB/sec. On my 3TB drive, that's supposed to be 4.3 hours. Concurrently, I'm runniung cat /dev/sdb > /dev/sda in another terminal. When I queried it's speed via cat /proc/[pid]/io, I calculated it to be roughly 180MB/sec.

This simultaneous cloning is working using the motherboard RAID in JBoD mode.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.