4

I'm thinking about using btrfs functions for both deduplicate and compress my storage...

My question is: is it efficient?

I mean, does it deduplicate AND compress at the same time efficiently?!

I'm using Ubuntu 14.04 64-bit. Also, I'm planning to use it with raid0 on top of EBS-Optimized volumes in a Cloud Computing environment...

Tks! Thiago

Improve this question
1
  • I'd not put anything that I am not willing to lose on btrfs at this time....not mature enough.
    – mdpc
    May 31, 2014 at 22:48

2 Answers 2

Reset to default
5

If you are writing about the deduplication with the bedup utility, then the answer is: deduplication and compression are completely independent processes.

During the deduplication, no file contents are rewritten (only metadata), so there is no opportunity to compress the data.

And I am 99% sure, that compression status is transparent to the bedup since it is next to impossible to get the original, compressed contents of the file out of the btrfs.

On the other hand, recompression of old data happens when you defragment the drive (e.g. with btrfs filesystem defragment -r -v -clzo /). This will currently break the de-duplication of files.

Improve this answer
3
  • Defragmentation undeduplicates (re-duplicates?) the data though, doesn't it? Because the cow-ed files are no longer cow-ed, if I understand correctly. Or are you saying this changed with 3.9?
    – Jo-Erlend Schinstad
    Jun 1, 2014 at 9:54
  • After kernel 3.9 defregmentation no longer re-duplicate files. Or so it is says - I never tested this aspect myself. The particular kernel patch (git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/…) says only about subvolumes. It seems that defragmentation inside a subvolume never resulted in reduplication of deduplicated contents.
    – Adam Ryczkowski
    Jun 2, 2014 at 10:55
  • 1
    Ah! That's really nice! :) I find it slightly difficult to keep up with Btrfs development. :)
    – Jo-Erlend Schinstad
    Jun 6, 2014 at 14:05
2

There is no online deduplication currently. It's planned, but not yet implemented. There is an experimental patchset ( www.mail-archive.com/linux-btrfs%40vger.kernel.org/msg32862.html ) if you want to try it.

Just from personal experience there is no noticeable performance hit of online compression on a hasswell i5 4460 when writing to a ssd and 3 hdds at once (one of the hdds is formated with xfs so no compression was being performed. Defragging/recompressing the 3 btrfs drives gives a cpu load around 50% but the system still feels smooth. This is with lzo and compression not forced (unless you force it btrfs stops trying to compress a file if it's incompressible + lzo is faster than zlib). The limit is still drive speed, even with a ssd.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .