1

I have a Ubuntu VM which has a volume containing some very, very compressible datasets on it.

For this reason, I've converted the volume in question to a ZFS volume, because I can then use ZFS compression.

This is all working really well, but I'm confused by some of the ZFS status output.

durr@graphical:/tank$ du . -h --max-depth=1; echo -----; du . -h --apparent-size --max-depth=1
1.9G    ./carbon
1.9G    .
-----
193G    ./carbon
193G    .

Note: /tank/ is the mountpoint of the ZFS volume.

So, give the above, I'm currently getting ~1% compression ratio (this is expected, the volume is almost entirely Carbon data files, which are mostly empty, so they should be extremely compressible).

However, if I ask ZFS about the volume:

durr@graphical:/tank$ sudo zfs get all tank
NAME  PROPERTY              VALUE                  SOURCE
tank  type                  filesystem             -
tank  creation              Mon Dec 25  7:27 2017  -
tank  used                  1.87G                  -
tank  available             239G                   -
tank  referenced            1.85G                  -
tank  compressratio         4.39x                  -
tank  mounted               yes                    -
tank  quota                 none                   default
tank  reservation           none                   default
tank  recordsize            128K                   default
tank  mountpoint            /tank                  default
tank  sharenfs              off                    default
tank  checksum              on                     default
tank  compression           on                     local
tank  atime                 on                     default
tank  devices               on                     default
tank  exec                  on                     default
tank  setuid                on                     default
tank  readonly              off                    default
tank  zoned                 off                    default
tank  snapdir               hidden                 default
tank  aclinherit            restricted             default
tank  canmount              on                     default
tank  xattr                 on                     default
tank  copies                1                      default
tank  version               5                      -
tank  utf8only              off                    -
tank  normalization         none                   -
tank  casesensitivity       sensitive              -
tank  vscan                 off                    default
tank  nbmand                off                    default
tank  sharesmb              off                    default
tank  refquota              none                   default
tank  refreservation        none                   default
tank  primarycache          all                    default
tank  secondarycache        all                    default
tank  usedbysnapshots       0                      -
tank  usedbydataset         1.85G                  -
tank  usedbychildren        18.7M                  -
tank  usedbyrefreservation  0                      -
tank  logbias               latency                default
tank  dedup                 on                     local
tank  mlslabel              none                   default
tank  sync                  standard               default
tank  refcompressratio      4.40x                  -
tank  written               1.85G                  -
tank  logicalused           8.13G                  -
tank  logicalreferenced     8.13G                  -
tank  filesystem_limit      none                   default
tank  snapshot_limit        none                   default
tank  filesystem_count      none                   default
tank  snapshot_count        none                   default
tank  snapdev               hidden                 default
tank  acltype               off                    default
tank  context               none                   default
tank  fscontext             none                   default
tank  defcontext            none                   default
tank  rootcontext           none                   default
tank  relatime              on                     temporary
tank  redundant_metadata    all                    default
tank  overlay               off                    default

ZFS is reporting a compression ratio of either 4.39x or 4.40x, depending on where you look. However, with the ~1% compression ratio from earlier, I'd expect to see either 0.01x or 99.0x, depending on how ZFS represents it's status.

Googling around, I can't seem to find the documentation on the compressratio member. It definitely changes as you move data around, because I've seen it vary, but what is it actually telling me?


Thinking about it, I also have ZFS deduplication turned on for this volume, so I thought it could be deduplicating the empty blocks. However, that doesn't seem correct:

durr@graphical:/tank$ sudo zpool get all tank
NAME  PROPERTY                    VALUE                       SOURCE
tank  size                        248G                        -
tank  capacity                    0%                          -
tank  altroot                     -                           default
tank  health                      ONLINE                      -
tank  guid                        11995166271724776732        default
tank  version                     -                           default
tank  bootfs                      -                           default
tank  delegation                  on                          default
tank  autoreplace                 off                         default
tank  cachefile                   -                           default
tank  failmode                    wait                        default
tank  listsnapshots               off                         default
tank  autoexpand                  off                         default
tank  dedupditto                  0                           default
tank  dedupratio                  1.12x                       -
tank  free                        246G                        -
tank  allocated                   1.69G                       -
tank  readonly                    off                         -
tank  ashift                      0                           default
tank  comment                     -                           default
tank  expandsize                  -                           -
tank  freeing                     0                           default
tank  fragmentation               1%                          -
tank  leaked                      0                           default
tank  feature@async_destroy       enabled                     local
tank  feature@empty_bpobj         enabled                     local
tank  feature@lz4_compress        active                      local
tank  feature@spacemap_histogram  active                      local
tank  feature@enabled_txg         active                      local
tank  feature@hole_birth          active                      local
tank  feature@extensible_dataset  enabled                     local
tank  feature@embedded_data       active                      local
tank  feature@bookmarks           enabled                     local
tank  feature@filesystem_limits   enabled                     local
tank  feature@large_blocks        enabled                     local

I have no idea where the extra data is, from ZFS's perspective. I think the files are sparse. Does ZFS not dedicate the disk-space to sparse files immediately?

1
  • I guess this is due to that sparse holes are not considered "compressed" and null pages will be mapped to sparse holes when compression is enabled (source of info). So what you are seeing as compressratio is the real compressed data, minus sparse files and null files. – Thomas Dec 30 '17 at 11:37
2

It looks like ZFS is turning a null file into a sparsefile when compression is enabled. Taken from the comment of DeHackEd from here.

The most likely answer to your question is, sparse holes are not considered "compressed". They are holes. You get the same thing on ext4 and it doesn't support compression at all. ZFS will turn all-null pages into sparse holes when compression is enabled.

I also created some files on a ZFS dataset using a sparsefile, a file created from /dev/zero and a file created only with the character a to get a good compression.

Commands used to create the files.

  • truncate -s $((1024*1024*1024)) /tank1/sparsefile
  • dd if=/dev/zero of=/tank1/zerofile bs=1073741824 count=1
  • used some for loops to echo a into the afile

First check the compressratio on the empty fileset tank1.

[root@localhost tank1]# zfs get all tank1 | grep compress
tank1  compressratio         1.00x                  -
tank1  compression           lz4                    local
tank1  refcompressratio      1.01x                  -

Then create a sparsefile and a file from /dev/zero with a size of 1GB and check the compressratio again.

[root@localhost tank1]# truncate -s $((1024*1024*1024)) sparsefile
[root@localhost tank1]# dd if=/dev/zero of=/tank1/zerofile bs=1073741824 count=1

[root@localhost tank1]# zfs get all tank1 | grep compress
tank1  compressratio         1.00x                  -
tank1  compression           lz4                    local
tank1  refcompressratio      1.01x                  -

Nothing has changed, although the zerofile should be considered as pretty good compressible. When using sparsefiles, space is never allocated right now but only on demand. This is the behaviour on any filesystem as this is filesystem independent.
All what is done is to set the Size parameter, but does not allocate any blocks as you can see from stat.

[root@localhost tank1]# stat sparsefile 
  File: ‘sparsefile’
  Size: 1073741824  Blocks: 1          IO Block: 131072 regular file
Device: 2ah/42d Inode: 2           Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2017-12-30 15:31:37.512845721 +0100
Modify: 2017-12-30 15:31:37.513845720 +0100
Change: 2017-12-30 15:31:37.513845720 +0100
 Birth: -

[root@localhost tank1]# stat zerofile 
  File: ‘zerofile’
  Size: 1073741824  Blocks: 1          IO Block: 131072 regular file
Device: 2ah/42d Inode: 3           Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2017-12-30 15:31:41.742838662 +0100
Modify: 2017-12-30 15:31:42.616837203 +0100
Change: 2017-12-30 15:31:42.616837203 +0100
 Birth: -

So the sparsefile and the zerofile look pretty much the same and have only 1 block allocated.
If we do the same on an ext4 filesystem, we can see the difference as the blocks for the zerofile are allocated.

[root@localhost test]$ stat sparsefile
  File: sparsefile
  Size: 1073741824  Blocks: 0          IO Block: 4096   regular file
Device: fd02h/64770d    Inode: 2883724     Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1000/    root)   Gid: ( 1000/    root)
Access: 2017-12-30 15:53:46.477442716 +0100
Modify: 2017-12-30 15:53:46.477442716 +0100
Change: 2017-12-30 15:53:46.477442716 +0100
 Birth: -

[root@localhost test]$ stat zerofile
  File: zerofile
  Size: 1073741824  Blocks: 2097160    IO Block: 4096   regular file
Device: fd02h/64770d    Inode: 2884453     Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1000/    root)   Gid: ( 1000/    root)
Access: 2017-12-30 15:54:11.014403727 +0100
Modify: 2017-12-30 15:54:11.311403254 +0100
Change: 2017-12-30 15:54:11.311403254 +0100
 Birth: -

Now let's look at an example with a file, containing only the character a with a size of 1GB on ZFS.

[root@localhost tank1]# du -h afile
33M afile
[root@localhost tank1]# du -h afile --apparent-size
1.0G    afile

[root@localhost tank1]# zfs get all tank1 | grep compress
tank1  compressratio         31.16x                 -
tank1  compression           lz4                    local
tank1  refcompressratio      31.89x                 -

Pretty good compression ratio and it does differ from a file containing zeros.

2
  • Cool! Also, somehow I didn't know about the stat command. The only comment I have is that the quoted comment about ext4 sparsefiles doesn't seem correct, as I moved from ext4 to ZFS because ext4 was allocating the entire sparse file at creation. Perhaps it's because the software that creates the file doesn't create the sparse-file in a way ext4 expects, and only ZFS is smart enough to figure out that it is actually a sparse file. – Fake Name Dec 31 '17 at 0:03
  • Concerning the sparsefile, if the space is allocated at file creation, then it is not a sparse file. A sparse file is not a specific type of file, but rather a method to allocated or save space. – Thomas Dec 31 '17 at 8:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.