10

If I make a disk image and compress it with gzip/xz/etc is there a way to mount it directly without first uncompressing it?

Say I've used

sudo dd if=/dev/sdc1 | gzip -9 > image1.dd.gz

how can I mount the original image, without creating an uncompressed copy first?

Or I've used

sudo dd if=/dev/sdc | gzip -9 > wholedisk.dd.gz

and the disk has multiple partitions, would that make it any harder?

With an uncompressed image of a whole disk then using kpartx or newer versions of losetup with it's -P flag should create a loop for each partition.

But is there a way to mount/losetup/read the compressed image?

If it won't work for gzip/xz, is there any compression method this would work for?


Note on duplicate Q

The currently suggested duplicate

DOES NOT USE COMPRESSION, and IS NOT A DUPLICATE.

mount will not mount a compressed image by itself.

  • I can tell you that it definitely is possible. But I can't tell you how atm because the instructions are on 2 external HDD and right now I have access to neither but I want to let you know that it works. I'll come back with an answer when I have one (and no one else posted one until then). – UTF-8 Oct 12 '16 at 20:03
  • 3
    Possible duplicate of Mount single partition from image of entire disk (device) – Panther Oct 12 '16 at 20:20
  • 1
    @UTF-8 That's encouraging, thanks. FYI, original goal is to make a smaller backup of a drive/partition that's nearly empty or has lots of easily compressible data, and be able to read it without writing a LOT of extra data or needing a ton of free space to extract it – Xen2050 Oct 12 '16 at 20:20
  • See also major.io/2010/12/14/… – Panther Oct 12 '16 at 20:21
  • 1
    @bodhi.zazen Pardon? Maybe I should clarify, mount does not automatically uncompress images, so this Q is not a duplicate of askubuntu.com/questions/69363 (maybe I missed the joke, hopefully uncompressing isn't the only answer :) – Xen2050 Oct 12 '16 at 20:40
9

You can use squashfs to compress disk images and then mount them.

Create the disk image

If you haven't got a disk image yet use dd to create one:

dd if=/dev/sda1 of=/tmp/sda1.img bs=4M

Compress the image with squashfs

Install squashfs:

apt-get install squashfs-tools

Compress the image:

mksquashfs /tmp/sda1.img /tmp/sda1.squash.img

Or Stream the compression (don't need a temporary dd file)

mkdir empty-dir
mksquashfs empty-dir squash.img -p 'sda_backup.img f 444 root root dd if=/dev/sda bs=4M'

Compliments to terminator14 at UbuntuForums.org. Definitions/Explanations:

  • empty-dir - "source" dir. Basically in our case, just an empty dir to satisfy mksquashfs' input arg format
  • squash.img - the destination and filename of the output squashfs file
  • sda_backup.img - the name of the dd backup INSIDE the squashfs file
  • f - specifies that sda_backup.img is a regular file (as opposed to a directory, block device, or char device)
  • 444 - permissions of the sda_backup.img file inside the squashfs image
  • root root - UID and GID for the sda_backup.img file inside the squashfs image. Can be specified by decimal numbers, or by name
  • dd if=/dev/sda bs=4M - the dd command used to read the device we want backed up

Mount the image

First mount the squashfs image:

mkdir /mnt/squash
mount /tmp/sda1.squash.img /mnt/squash

This will present the un-compressed disk image for you to mount:

mkdir /mnt/sda1
mount /mnt/squash/sda1.img /mnt/sda1

Or if it's a full drive image (partitioned) you could use losetup to attach the dd image to a loop device (possibly optional) and then kpartx -a or partprobe to find & separate the partitions to separate devices, or even vgscan / vgchange -ay if there's LVM.

  • 2
    I added some on-the-fly squashfs info, to avoid needing a temporary plain dd file. Great answer with squashfs, just doesn't do the more pipe-friendly xz/gz formats – Xen2050 Dec 11 '16 at 11:25
  • Would that somehow work with a full disk image containing multiple partitions as well? – friederbluemle Nov 18 '17 at 17:43
5

I was able to get an NBD server to "serve up" an un-compressed version of a .gz or .xz file. I'm not sure if it has to read the entire compressed file very often (only tested so far with small files in ram) but at least it works without having to extract a whole un-compressed copy.

Viewing a gz / xz Compressed dd image "on-the-fly"

  1. Install nbdkit & client

    apt install nbdkit nbd-client
    
  2. Create demo "dd" style file - an empty 50M file with an ext filesystem & zip it (Optional for testing only)

    fallocate -l 50M 50m
    mkfs.ext3 -v 50m
    

    Then compress with either gzip or xz

    gzip -vk9 50m
    xz -v9k 50m
    

    Note: An xz option of --block-size=16MiB should give better random access performance, but at the expense of less compression (maybe 1%?). See the nbdkit's xz plugin description.

  3. Run server on COMPRESSED image

    nbdkit -v --no-fork -i 127.0.0.1 /usr/lib/i386-linux-gnu/nbdkit/plugins/nbdkit-gzip-plugin.so file=~/Downloads/50m.gz
    

    or for xz

    nbdkit -v --no-fork -i 127.0.0.1 /usr/lib/i386-linux-gnu/nbdkit/plugins/nbdkit-xz-plugin.so file=~/Downloads/50m.xz
    

    The --no-fork tells the server "Don't fork into the background" so seeing possible error messages & killing it with CTRL-C is a little easier.

    -v may / will show too many status messages, especially when reading.

  4. Run client (in another terminal/window)

    sudo nbd-client -nofork 127.0.0.1 10809 /dev/nbd0
    

    The -nofork is similar to -no-fork above, but I could not kill the client with CTRL-C, kill <pid>, or sudo kill <pid>. Ending the server with CTRL-C stopped both server & client.

  5. Observe that /dev/nbd0 is now identical to UN-COMPRESSED image

    $ sudo diff -s /dev/nbd0 ~/Downloads/50m
    Files /dev/nbd0 and /home/user/Downloads/50m are identical
    

    /dev/nbd0 can now be used just like the uncompressed image; mounted, read/copy with dd, or use kpartx, partprobe, vgscan / vgchange, etc!

Thanks to Cristian Ciupitu on Unix & Linux, though it took a little more digging to find the plugin & get everything going together.

0

Here is my way of doing it , dont know the console equivalent of this though,

  • Compress the folder into an iso. enter image description here
  • Mount the .iso file with Disk Image Mounter. enter image description here
  • Tada I open the compressed iso without unpacking it. enter image description here
  • 3
    Does this method actually compress the data? It looks like it uses a plain iso9660 image (via file-roller?), which does not use compression. I tried it on a 100MB file of all zero's (dd if=/dev/zero and file-roller) and it actually adds 362k of data (iso overhead I suppose). Thanks though, the idea is on the right track – Xen2050 Oct 22 '16 at 12:07
  • Sorry but I dont have any idea on compression technologies but if you try any other compression format , i bet you can also list that with file-roller, but if you want to read that file you need to extract only that specific file. – hellozee Oct 22 '16 at 13:33

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.