I have huge data sets to deal with, and so my 1TB drive is filling up quickly. However, the data is highly compressible when using LZMA (7zip, bzip2). The only file system with transparent compression I've discovered so far is Btrfs, which is not assumed stable yet. Is there a reliable solution for piping the data through a compression and do a bind-mount to a compressed image file?

Any idea is welcome!

I do not want to patch any file system driver though.



You can also try ZFS with Gzip-9 with would give some good results.

Squashfs may also be considered (I didn't try it though).

  • Is ZFS considered a stable option with high performance? ASFAIK it is only avaible as FUSE module... – Andre Dec 2 '12 at 22:24

Fusecompress would have been a ideal, since that lives in virtual filesystem and thus has no requirement on changing the actual filesystem run. However, sadly, the project is dead upstream and I would recommend to stay away from it for this reason.

BtrFS supports only ZLIB and LZO compression at the time of writing.

ZFS-on-Linux (non-FUSE) offers more options, including LZ4 compression. It's actively maintained and the userbase for ZFS in general is quite large in other than-Linux Unices (Sun, BSD, etc.).

Migrating to a "3rd generation" filesystem could be a good idea for other reasons too, but that's totally up to how flexible you are in implementing just this.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.