0

I have a 6.3G filesystem dump that I've been trying to compress using squashfs, but after using unsquashfs, the generated file is different from the original dump.

I generated a difference between the original file and the file in the squashfs system with the following command:

cmp -l /media/daboross/extstorage/backup-2014-12-05/sda7_backup.img \
    /media/daboross/disk/sda7_backup.img  \
    | gawk '{printf "%08X %02X %02X\n", $1, strtonum(0$2), strtonum(0$3)}'  \
    > diff.txt

where /media/daboross/extstorage/backup-2014-12-05/sda7_backup.img is the original file and /media/daboross/disk/ is a mount of the squashfs file. See https://superuser.com/a/125408 for the source of this command.

Here's the generated diff file: http://sprunge.us/BFPS. There might be some pattern in this that can help diagnose the problem, I'm not sure what it would be, but including this just in case.

If there's any information that would be helpful, feel free to ask for it. I don't currently have the squashfs filesystem that generated that that difference file was generated from, but I can just run mksquashfs again if need be - it is consistently creating a file which isn't the same.

The version of mksquashfs/unsquashfs (from squashfs-tools):

mksquashfs version 4.2-git (2013/04/07)
unsquashfs version 4.2-git (2013/03/13)

Any ideas on why this occurs or what I can do to help are greatly appreciated.

Edit/Update:

After upgrading from the version of squashfs-tools packaged with Ubuntu to squashfs v4.3, I still get an error, but not as much of one.

Running the above binary diff command produces this for the file compressed and decompressed using v4.3: http://sprunge.us/DDAM.

If you look at the binary diff files you see that the diff file produced with v4.3 is exactly the same as the last part of the file produced with 4.2-git. I would guess this probably means that part of the corruption was fixed in v4.3, but not all of it.

2
  • Have you tried mounting both file system dumps and comparing the source files inside them? Because it might just be a timestamp that is propagating through the images... (just a hunch)
    – Fabby
    Dec 7, 2014 at 19:49
  • I haven't tried mounting both of them - but the squashfs is directly created from the filesystem dump image. Because I haven't mounted it, I don't see how a timestamp could have change d :/
    – daboross
    Dec 8, 2014 at 1:08

1 Answer 1

1

Obvious point - you complain that Unsquashfs is generating a different file, but then demonstrate this by mounting the Squashfs filesystem and comparing that....

If the bug was in Unsquashfs you would expecting mounting the filesystem to work (giving the correct output). If both Unsquashfs and mounting doesn't work, then the bug (if any) is likely to be in Mksquashfs.

The version of mksquashfs/unsquashfs (from squashfs-tools):

mksquashfs version 4.2-git (2013/04/07)

unsquashfs version 4.2-git (2013/03/13)

These look like snapshots from my git development tree (they have -git in the version).

It is possible they have been snapshotted when the development code was in an inconsistent state or had temporary bugs.

You should try the release versions of squashfs-tools. Squashfs-tools 4.3 was released this year and you should try that. This however has a known bug causing incorrect filesystems in certain cases (introduced in Sept 2011), but this doesn't look like the bug you're hitting.

Depending on whether the above Squashfs-tools 4.3 works, you should also try the earlier release Squashfs-tools 4.2 (from 2011), and the latest development version from git (where the one known filesystem corruption bug has been fixed).

The release tarballs and the git development tree can be got from

http://sourceforge.net/projects/squashfs

4
  • I've tried using unsquashfs and mounting the filesystem - both result in incorrect files. I'll try upgrading to the release version, thank you for suggesting that! Previously I just had the one that shipped with Ubuntu 14.04 (packages.ubuntu.com/trusty/squashfs-tools).
    – daboross
    Dec 8, 2014 at 1:10
  • Using v4.3 produced a strange result - the majority of the corruption didn't happen, but the last bit of data changed is changed in the exact same way as 4.2-git. I'm going to try the development version now.
    – daboross
    Dec 8, 2014 at 1:42
  • Awesome, using the development version worked!
    – daboross
    Dec 8, 2014 at 2:02
  • 2
    Good to know the development version works, this means you're hitting the one known corruption bug. Sod's law, this bug was in the development tree for 2 1/2 years without triggering, and it made it through testing into the next release. The cause of the other corruption you're seeing is unknown, but may be due to an unstable snapshot from the devel tree (I don't really like distros using snapshots from my development tree in their releases, and I wasn't aware this had happened here). When I get time I'll download their package and see where/what the breakage is. Dec 8, 2014 at 16:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .