2

I created yesterday a drive image using GZip.

I opened the 51.3 GiB file named ssd.img.gz to see one icon thats a package of 3.9 GiB, yes, thats THree point Nine Gigabytes in size and has the name of the image as ssd.img

That does not seem right to me, has the image creation worked or not?

Yes the drives have data and an operating system on it.

Should I have used cat instead? only I did and it kept on saying that it had nothing to do.

Would using the | zip instead be a better option?

** EDIT **

The command used was:

mark@zotac:~$ sudo -l
Enter password:<entered password to get in to root>
root@zotac:~# dd if=/dev/sda | gzip -c > "/media/mark/Seagate External Drive/ssd.img.gz

the terminal spent the best part of 10 Hours at building the image, something in itself that I also found suspect, why so log when copying the files to a folder that I wanted, some 40GiB took 45 minutes...

The source drive /dev/sda had no mounted partitions.

** EDIT **

Just to clarify, I am trying to make an image of the drive to back up later if things go wrong with a repair, the drive itself needs to be a file on a drive as the target drive partition is using almost all of the drive, the partitions can't be resized on it without formatting it and I am not about to lose 1.4TB of data.

So the dd if=/dev/sda of=/<target> isn't what I need, what I do need is an image of the drive that I can restore the image from to the drive should the repair go wrong.

So what command do I use, dd or cat? Why does | zip keep telling me it has nothing to do when used with cat but no error or warning with dd

  • 1
    Please tell us the exact command you used to create the image. – Rmano Mar 2 '15 at 12:28
  • What does zcat ssd.img.gz | wc -c return? – David Foerster Mar 2 '15 at 12:31
  • The terminal just locks, I ^c to kill the operation as nothing was happening. – Mark Giblin Mar 2 '15 at 12:48
  • 1
    @DavidFoerster, counting ONE BY ONE 64 billions chars will be a operation lasting days. What's wrong with ls -l? – Rmano Mar 3 '15 at 13:32
  • ls -l can tell me the unpacked size of compressed files? – David Foerster Mar 3 '15 at 21:21
2

If the disk is almost full, there is no sense in compressing the image; it will take ages with no practical effect. And restoring will be much slower too.

dd copies physical, byte-by-byte partitions or disks. It works by reading N bytes from the input and writing them to the output, and then repeating. N is by default 512 bytes.

To copy a disk/partition physically, use

dd bs=10M if=/dev/sda of=/path/to/dest/image.img 

this will read blocks of 10Mbytes at a time. If you have plenty of RAM you can up that number too. The size of the file will be exactly the same as the size of the disk/partition used (but it can occupy less disk sometime --- dd can create sparse files).

You can check the progress in another terminal with

ls -lh /path/to/dest/image.img 

which will give you the size of the image file (more or less --- it will be in chunks, really).

Triple check (with mount) that no partition of the disk to copy is mounted or used anywhere, or the created image will be invalid (and worst things can happen, too). Be careful about swap areas, too.

You can mount the image to check if it's ok with a loopback filesystem and a bit of tricks.

Anyway, there are also tools that helps the tasks: How to make a disk image and restore from it later?

If your system is not recent (the system from where you are booting, I mean), be warned of this bug --- fixed from 14.04 onward, but still there for older kernels.

  • I will try it, it may be hours before I can attempt your suggestion and tomorrow before I can test the outcome. As the drive is 512k blocks, I think that it would be safer to stick to the default value for block size. The install is a 14.10, so its current. – Mark Giblin Mar 3 '15 at 14:15
  • dd default is 512 bytes... – Rmano Mar 3 '15 at 15:38
  • So far, no errors!!! – Mark Giblin Mar 3 '15 at 16:07
0

Compressing a whole drive's or a partition's image using Gzip may lead to a tremendous reduction in file size but only in case there were no unused data remnants stored and unused space is uniformly filled e.g. by Zeros.

This may be the case for a very new hardly used drive, or for a drive where we had manually filled unused space with Zeros (e.g. using the utility zerofree Install zerofree).

  • This is however not the case for an old used drive where data remnants will be randomly dispersed all over the drive, for a drive where a secure wipe had filled unused space with random numbers, or after a wholde disk encryption. In these cases Gzip will not be able to compress so much or not at all (random numbers can't be compressed).

  • It also is not the case for a SSD used in write mode for some time because on each write all memory blocks will be evenly distributed all over the drive by the drive's firmware to avoid wear leveling. Then any compression of an image will also compress those data remnants which will be more and more random over time.

So, in your case a 3.9 GiB compressed image of a 50 GiB SSD may be possible for a rather new, hardly used drive or a drive mainly used for reading but not for writing much onto it. But then it is a sensible value.

To find out for sure we'd need to decompress the image to compare it's content with the source.

  • Can I just take an uncompressed image, I have enough drive space for the 64GiB SSD on an external USB device. I only go by the examples presented from searches, so I have no idea if I need to add something or take it away from the command line. I would prefer a ZIP file. – Mark Giblin Mar 2 '15 at 13:40
  • Yes, you don't need to compress your image if space is not an issue - the copying will be much faster. – Takkat Mar 2 '15 at 13:48
  • So how do I save the image in a format like ZIP thats not compressed, the drive is about 98% full, I have 500+Gigs on the USB drive, so space is not an issue but preserving the drive as it currently stands even including its bad sector, the reason it won't boot properly, need the image in case anything goes wrong and I can have a second stab at it. So what do I do to the | gzip -c > part of the string? do I | zip > ? – Mark Giblin Mar 2 '15 at 13:55
  • 1
    You should unmount the drive to clone from, mount the drive to clone to, then (e.g. from a live session) sudo dd if=/dev/sdX of=/path/to/image.img - replace paths with your's. Take care! dd can destroy everything if locations were bad. Also see askubuntu.com/questions/19901/… – Takkat Mar 2 '15 at 14:29
  • I was in root in a system I am not copying from, I think that this is what confusing people, the drive copy from and copy to are on the same system I am booted on but the drive I am booted on is separate SDCard, the source is unmounted the target is mounted the command was issued whilst I was rooted and I tried a LiveCD/USB but they were denied permission, no idea why but have had no end of problems with this 14.10 install and L-CD. So I plod on and I have had to install 14.10 again! only on a larger SDCard as making an image of the original wasn't working. So is dd & cat broke? – Mark Giblin Mar 2 '15 at 15:17

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.