I have a Ubuntu VM which has a volume containing some very, very compressible datasets on it.
For this reason, I've converted the volume in question to a ZFS volume, because I can then use ZFS compression.
This is all working really well, but I'm confused by some of the ZFS status output.
durr@graphical:/tank$ du . -h --max-depth=1; echo -----; du . -h --apparent-size --max-depth=1 1.9G ./carbon 1.9G . ----- 193G ./carbon 193G .
/tank/ is the mountpoint of the ZFS volume.
So, give the above, I'm currently getting ~1% compression ratio (this is expected, the volume is almost entirely Carbon data files, which are mostly empty, so they should be extremely compressible).
However, if I ask ZFS about the volume:
durr@graphical:/tank$ sudo zfs get all tank NAME PROPERTY VALUE SOURCE tank type filesystem - tank creation Mon Dec 25 7:27 2017 - tank used 1.87G - tank available 239G - tank referenced 1.85G - tank compressratio 4.39x - tank mounted yes - tank quota none default tank reservation none default tank recordsize 128K default tank mountpoint /tank default tank sharenfs off default tank checksum on default tank compression on local tank atime on default tank devices on default tank exec on default tank setuid on default tank readonly off default tank zoned off default tank snapdir hidden default tank aclinherit restricted default tank canmount on default tank xattr on default tank copies 1 default tank version 5 - tank utf8only off - tank normalization none - tank casesensitivity sensitive - tank vscan off default tank nbmand off default tank sharesmb off default tank refquota none default tank refreservation none default tank primarycache all default tank secondarycache all default tank usedbysnapshots 0 - tank usedbydataset 1.85G - tank usedbychildren 18.7M - tank usedbyrefreservation 0 - tank logbias latency default tank dedup on local tank mlslabel none default tank sync standard default tank refcompressratio 4.40x - tank written 1.85G - tank logicalused 8.13G - tank logicalreferenced 8.13G - tank filesystem_limit none default tank snapshot_limit none default tank filesystem_count none default tank snapshot_count none default tank snapdev hidden default tank acltype off default tank context none default tank fscontext none default tank defcontext none default tank rootcontext none default tank relatime on temporary tank redundant_metadata all default tank overlay off default
ZFS is reporting a compression ratio of either
4.40x, depending on where you look. However, with the ~1% compression ratio from earlier, I'd expect to see either 0.01x or 99.0x, depending on how ZFS represents it's status.
Googling around, I can't seem to find the documentation on the
compressratio member. It definitely changes as you move data around, because I've seen it vary, but what is it actually telling me?
Thinking about it, I also have ZFS deduplication turned on for this volume, so I thought it could be deduplicating the empty blocks. However, that doesn't seem correct:
durr@graphical:/tank$ sudo zpool get all tank NAME PROPERTY VALUE SOURCE tank size 248G - tank capacity 0% - tank altroot - default tank health ONLINE - tank guid 11995166271724776732 default tank version - default tank bootfs - default tank delegation on default tank autoreplace off default tank cachefile - default tank failmode wait default tank listsnapshots off default tank autoexpand off default tank dedupditto 0 default tank dedupratio 1.12x - tank free 246G - tank allocated 1.69G - tank readonly off - tank ashift 0 default tank comment - default tank expandsize - - tank freeing 0 default tank fragmentation 1% - tank leaked 0 default tank feature@async_destroy enabled local tank feature@empty_bpobj enabled local tank feature@lz4_compress active local tank feature@spacemap_histogram active local tank feature@enabled_txg active local tank feature@hole_birth active local tank feature@extensible_dataset enabled local tank feature@embedded_data active local tank feature@bookmarks enabled local tank feature@filesystem_limits enabled local tank feature@large_blocks enabled local
I have no idea where the extra data is, from ZFS's perspective. I think the files are sparse. Does ZFS not dedicate the disk-space to sparse files immediately?