Short summary:
When creating a single zfs disk pool consisting of only one disk with a capacity of 1TB (= 931GiB) the file system only showed 899 GiB free space (df -h
or zfs list
; zpool list
actually showed the partition size (931 GiB) minus some overhead (resulting in 928 GiB space).
Longer version:
I was trying to setup a zfs disk pool consisting of only one disk with a capacity of 1TB (= 931,53 GiB):
# fdisk -l /dev/sdb
Disk /dev/sdb: 931.53 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: xxxx
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: xxx
Device Start End Sectors Size Type
/dev/sdb1 2048 1953516976 1953514929 931.5G Linux filesystem
When setting up a zfs pool
# zpool create -f -o ashift=12 tank /dev/sdb1;
there are 32,5 GiB missing:
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
tank 360K 899G 96K /tank
What is causing a 32,5 GiB overhead on a single disk setup?
zpool list
is reporting:
# zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
tank 928G 444K 928G - - 0% 0% 1.00x ONLINE -
but this isn't the actually useable space since also df -h
is reporting:
# df -h
Filesystem Size Used Avail Use% Mounted on
tank 899G 128K 899G 1% /tank
There is no quota or reservation set:
# zfs get quota
NAME PROPERTY VALUE SOURCE
tank quota none default
# zfs get reservation
NAME PROPERTY VALUE SOURCE
tank reservation none default
# zfs get refquota
NAME PROPERTY VALUE SOURCE
tank refquota none default
# zfs get refreservation
NAME PROPERTY VALUE SOURCE
tank refreservation none default
# zfs get usedbyrefreservation
NAME PROPERTY VALUE SOURCE
tank usedbyrefreservation 0B -
Creating the zpool with ashift=9
doesn't make any difference.
I can live with an actual overhead of 3,5 GiB (= partition size vs. zpool reported size), but not with an overhead of 32,5 GiB or 29 GiB (= zfs reported size - zpool reported size) on such a small disk.
When trying the same with btrfs I get more free space:
# btrfs filesystem show
Label: 'tank' uuid: xxx
Total devices 1 FS bytes used 128.00KiB
devid 1 size 931.51GiB used 2.02GiB path /dev/sdb1
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 932G 3.8M 930G 1% /mnt
more details
# zfs --version
zfs-0.8.3-1ubuntu12.5
zfs-kmod-0.8.3-1ubuntu12.5
# uname -a
Linux xxx 5.4.0-58-generic #64-Ubuntu SMP Wed Dec 9 08:16:25 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
# cat /etc/os-release | grep VERSION
VERSION="20.04.1 LTS (Focal Fossa)"
VERSION_ID="20.04"
VERSION_CODENAME=focal
# zfs list -o space
NAME AVAIL USED USEDSNAP USEDDS USEDREFRESERV USEDCHILD
tank 899G 88.5K 0B 24K 0B 64.5K
Update:
re-created with the command zpool create -oashift=12 tank /dev/sdb1
.
No difference:
# zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
tank 928G 432K 928G - - 0% 0% 1.00x ONLINE -
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
tank 336K 899G 96K /tank
Output of zdb tank | grep metaslab | tail -n 3
:
# zdb tank | grep metaslab | tail -n 3
loading concrete vdev 0, metaslab 115 of 116 ...
metaslab 114 offset e400000000 spacemap 0 free 8G
metaslab 115 offset e600000000 spacemap 0 free 8G
vdev 0 metaslabs 116 fragmentation 0%
Output of zdb | grep metaslab_shift
:
# zdb | grep metaslab_shift
metaslab_shift: 33