12

I like 'Disks' (gnome-disk-utility) and its ability to create backups of system partitions.

enter image description here

But at some point I read on the internet about a program that would create such images that are not as large as the whole partition, but only as large as the files on it. Now I cannot find it again.

Is there such an utility?

Improve this question
2
  • 3
    CloneZilla, perhaps?
    – muru
    Feb 23, 2015 at 13:01
  • 1
    As I see clonezilla can be run only as a live session. I have edited to specify that I am looking for a program that can be installed and run from a OS, I am using multiple ubuntus and I prefer this solution
    – user47206
    Feb 23, 2015 at 14:42

8 Answers 8

Reset to default
9

The following answer uses a TUI, not a GUI solution:

Depending on your use-case (I use it to make system back-ups) you can use CloneZilla Live. It can create:

  • sparse images,
  • compressed images of:
    • entire systems,
    • entire drives
    • simply individual partitions.

The main advantages are:

  • that it boots from:
  • comes with its own OS (based on Ubuntu)
  • it can make exact cold system back-ups that are easily restored.

On top of all these goodies, it can copy MBRs as well.

If a cold system back-up is not what you're looking for, you can still use the technology that CloneZilla is based upon and that is partclone.

Improve this answer
7
  • 1
    I have edited to specify that I am looking for a GUI program that can be installed and run from a OS, I am using multiple ubuntus and I prefer this solution
    – user47206
    Feb 23, 2015 at 14:43
  • how to install pyclonezilla? not one google entry for this question
    – user47206
    Feb 23, 2015 at 22:18
  • 1
    I was too quick to point the GUI out: It doesn't work. CLI only! I'll delete my answer after you've left a note that you've read this.
    – Fabby
    Feb 24, 2015 at 5:06
  • 1
    As there is no program, it seems, to fulfill both conditions: 1) Have a 'Disks'-like GUI; 2) Create disk-image as large as the space used, there is no point in deleting your answer. CLI answers to GUI questions are usually presented here and often accepted, with good reason. I was just trying to push for what I wanted.
    – user47206
    Feb 24, 2015 at 10:17
  • 1
    :D I would take a backup and then shrink... ;-)
    – Fabby
    Feb 25, 2015 at 19:46
5

Achieving smaller disk images is usually done by ways of not copying zero outed blocks from the initial file system in the image. The concept is called sparse space in files or puching holes in files, and your host file system has to support it. a bit command line but try this:

sudo apt-get install virt-manager qemu-utils

And then once it's install you can use :

qemu-img convert -f raw -O raw  /dev/sdX /home/<user>/Ubuntu.img

This will create a sparse ( zeros in the file aren't written) image of your disk. where -f = format, -O output format. sdX the disk ( can use partitions as well). The program has advanced features such as snapshots and different output formats (vdi, vmdk, etc). Your place where you save the image has to support sparse files, or else the size will be just as big, since zeros in the partition will be written to disk. And to make the most of it, do run a program such as sfill or zerofree found in apt on the partition, because free space does not always mean zero space, and you want to maximize the amount of zero space on the file system.

Bonus ninja points. Since linux thinks of disks as "files", then you could also use plain cp, with the condition that your output partition where you park the file knows sparse files.

sudo cp --sparse=always /dev/sdX /home/<user>/Ubuntu.img

Which will create a raw copy by means of cp..

Examples of filesystems that support sparse : ext4, btrfs, xfs, ntfs ( ntfs-3g however might not allow sparse writes).

Examples of filesystems that do not : fat32.

Improve this answer
10
  • I have edited to specify that I am looking for a GUI program that can be installed and run from a OS, I am using multiple ubuntus and I prefer this solution
    – user47206
    Feb 23, 2015 at 14:42
  • 1
    what would be the command to restore the image?
    – user47206
    Feb 23, 2015 at 17:49
  • 1
    Usually on server farms we don't have that much in GUI options and build scripts around said commands. This is why command line is usually more complete/powerful on Unix. Converting back is just as easy as inverting the parameters.. sudo cp /home/<user>/Ubuntu.img /dev/sdX, or if you fancy dd, sudo dd if=/home/<user>/Ubuntu.img of=/dev/sdX. Same thing with qemu-img convert.. If you say, want to plant a virtual machine on real hardware. When it writes back, it will write zeros to disk, thus you unsparse, as it needs to properly address the partition's geometry.
    – user283885
    Feb 23, 2015 at 21:25
  • 1
    I understand your point and I apologize that I am not aware of any GUI tools to suit your needs. I just thought that at least something second to that might be of use to you.
    – user283885
    Feb 23, 2015 at 21:57
  • 1
    Welcome to this side of the fence. The reason I pointed to server farms is because of how most development is done lately. For example, the qemu-img is an utility from a much larger project ( openstack, started by redhat, rackspace & nasa). You have these big companies that build software to suit their data centers and then you have guys like us who piggyback on that effort. Sure it works, but it's spartan at best. For now, the best I can do is share the know how with others.
    – user283885
    Feb 23, 2015 at 22:32
4

As there doesn't seem to be any program that all by itself would satisfy the two conditions:

1) Have a 'Disks'-like GUI;

2) Create disk-image as large as the space used,

I was thinking about a rather simple workaround, but one involving two GUI programs, very common in Ubuntu:

One would be the same 'Disks' (gnome-disk-utility), used to to create a backup image of a given partition, and the second would be Gparted, to shrink the partition as much as possible before creating its backup image, and to resize it back to original after that.

I see no inconvenient in a such operation: shrinking a partition down to minimum would only take a few minutes (about a minute for 10GB) and would create an empty space to the right. After making a backup image of the disk with 'Disks', using Gparted the partition can be resized back to the right into the empty space up to its initial state, and that would take only a few seconds.

Tested on a partition with an Ubuntu installation:

Improve this answer
2
  • 1
    As I've just received an upvote to my answer, reading if you ever found a perfect answer, and this seems to be the one! Why don't you accept this one?
    – Fabby
    Apr 11, 2016 at 12:51
  • I'm the author of the question and of the above answer. Shrinking a partition is a risky action before which a back-up is recommended. Using a shrinking-partition action to make the back-up gives a logical loop. A much safer solution for a ext4 partition is to use Disks to make an image of the entire partition and then use e2fsck -f -y xxx.img and resize2fs -M xxx.img to resize as said here.
    – cipricus
    May 5, 2021 at 16:40
2

Edit: This will not work well if files deleted from the filesystem are still present on the partition, so first it's necessary to wipe the free space on the target partition:

  1. Mount the target partition
  2. Run dd if=/dev/zero of=<path_to_partition_volume>/file.dd && rm <path_to_partition_volume>/file.dd
  3. Unmount the target partition

Then:

You can read the whole partition block by block with dd and output everything to a compression utility like gzip. Most likely any compression tool, even using the fastest/lowest compression settings, will compress all the free space during the process.

With gzip:

sudo dd if=/dev/sdX | GZIP=-1 gzip >/path/to/backup.gz

(GZIP=-1 temporarily sets the environment variable GZIP to -1 to achieve the fastest/lowest compression)

To restore a backup:

gzip -d /path/to/backup.gz >/dev/sdX

Improve this answer
3
  • What do you think about shrinking the partition with gparted before using Disks to backup it?
    – user47206
    Feb 24, 2015 at 15:00
  • @cipricus Better wait for someone else to say something about this, because i'm not sure, even tough i think it should be fine to do it that way either
    – kos
    Feb 24, 2015 at 15:23
  • @kos I found the ddrescue command you give produced an image quite effortlessly but it did not reduce the size of the file at all. I was imaging a NTFS partition to ext4, both of which seem to work with sparse files but the .img file size shown in Nautilus exactly matched the originating partition size. So it looks as though fiddling about with gParted might still be needed?
    – PhilJ
    Jul 17, 2016 at 17:12
1

How about using ddrescue with -S option?

ddrescue -S /dev/sdXY /<path_to_backup>/backup.img

This will clone each block but zero blocks found in block device /dev/sdXY to /<path_to_backup>/backup.img

Improve this answer
2
  • Sounds great. But what would be the command to restore the image?
    – user47206
    Feb 23, 2015 at 17:51
  • @cipricus I found out that this is actually highly system-dependant, and won't work on every distribution, so i added a simpler yet maybe just a little slower method to accomplish the same without downloading additional software, which is the method i actually use
    – kos
    Feb 23, 2015 at 19:07
0

It can't be used to make a copy to a smaller drive. When I have a 64 Gb drive that is only 10% filled with data. I would like to be able to copy that to a 6 Gb imagefile, that I can copy on its turn to a 16Gb drive and than expand it to 16 gb

The only possibility I know now is use GParted to shrink the original, than copy it and then resize both. A very risky operation, since you need to shrink and resize the original.

Improve this answer
0

You could use rsync to backup just the files on the partition to another partition. Or tar to create a compressed archive containing all files.

Both probably have (multiple) GUIs, but I've never used those myself.

Improve this answer
0

Clonezilla can actually be quite friendly. The live CD doesn't require any command line usage - I'd be quite happy to call it a GUI even though it doesn't use the mouse - and I think the defaults are OK for a lot of cases. The problem is something you didn't explicitly call out: it doesn't seem to be designed as something you can install on an arbitrary linux system. It's packaged for Debian / Ubuntu specifically, and the Live CD is the easiest way to run it.

Clonezilla / partclone use filesystem-aware cloning and use their own compressed image format.

There's another non-GUI possibility to generate simple (mountable) sparse files, that doesn't rely on detecting zero sectors.

virt-sparsify /dev/your-block-device backup.img

If /tmp is a ramdisk it will warn that it potentially needs more space for temporary files. So you might need to add the option --tmp /var/tmp.

Improve this answer
2
  • I found the virt-sparsify method does certainly reduce the size of the .img file but it requires a very large /tmp directory, roughly equal to the size of the partition being imaged. It gave out a warning message but when I tried the --tmp option shown above (varied for my setup) it gave me >unknown option `--tmp' I'm not sure now how to go about diverting the /tmp output elsewhere and any advice on that would be helpful to me and to future readers of this post.
    – PhilJ
    Jul 17, 2016 at 17:02
  • Specify virt-sparsify --version? The option is documented on mine (man virt-sparsify)... I can't swear I've used it, but at least my current /tmp is tmpfs. If you can use --in-place, that looks like it will avoid temporary storage. Finally you might as well try TMPDIR=/var/tmp virt-sparsify .... The docs for my version say that's how you have to set the tmp dir for versions prior to 1.28.
    – sourcejedi
    Jul 17, 2016 at 21:09

You must log in to answer this question.