How can I shrink a zfs volume on ubuntu 18.04?

Question

Given an installation on a single disk with zfs, how can I shrink the zfs partition/pool so that I can add a new partition to the disk? Other instructions I've seen indicate setting the volsize property on the pool, but doing so gives the following error: cannot set property for 'rpool': 'volsize' does not apply to datasets of this type

$ zpool list
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
bpool  1.88G   196M  1.68G        -         -      -    10%  1.00x    ONLINE  -
rpool   228G  10.9G   217G        -         -     2%     4%  1.00x    ONLINE  -

Ben Mares · Accepted Answer · 2024-04-01 13:57:15Z

Actually, it is possible to do it. Because it is possible to remove top level vdev from pool if space permits. So the trick is to:

add a temp disk/file vdev to the pool with smaller size, but is enough to hold all existing data (including snapshots etc)
remove the old vdev
possibly re-partition old vdev to smaller size, or replace with a smaller disk
add it back
remove the temp disk.

All files should remain intact.

To illustrate the steps:

Allocate the files

# fallocate -l 3G 3G_1.img
# fallocate -l 3G 3G_2.img
# fallocate -l 2G 2G_1.img
# fallocate -l 2G 2G_2.img

Create zfs with the 2 3G files with mirror. (my directory is /var/snap/lxd/common/lxd/disks)

# zpool create test3g mirror /var/snap/lxd/common/lxd/disks/3G_1.img /var/snap/lxd/common/lxd/disks/3G_2.img
# zpool status test3g
  pool: test3g
 state: ONLINE
  scan: none requested
config:

    NAME                                         STATE     READ WRITE CKSUM
    test3g                                       ONLINE       0     0     0
      mirror-0                                   ONLINE       0     0     0
        /var/snap/lxd/common/lxd/disks/3G_1.img  ONLINE       0     0     0
        /var/snap/lxd/common/lxd/disks/3G_2.img  ONLINE       0     0     0

errors: No known data errors
# zpool list test3g
NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
test3g  2.75G   111K  2.75G        -         -     0%     0%  1.00x    ONLINE  -

You can clearly see the test3g pool is 2.75G usable, it is in mirror.

Let's try to create dummy file inside. This to simulate your existing data. And you can verify your data is intact after the exercise.

# echo test > /test3g/test.txt
# cat /test3g/test.txt
test
#

Now attach another mirror of smaller size (e.g. 2G)

# zpool add test3g mirror /var/snap/lxd/common/lxd/disks/2G_1.img /var/snap/lxd/common/lxd/disks/2G_2.img
# zpool status test3g
  pool: test3g
 state: ONLINE
  scan: none requested
config:

    NAME                                         STATE     READ WRITE CKSUM
    test3g                                       ONLINE       0     0     0
      mirror-0                                   ONLINE       0     0     0
        /var/snap/lxd/common/lxd/disks/3G_1.img  ONLINE       0     0     0
        /var/snap/lxd/common/lxd/disks/3G_2.img  ONLINE       0     0     0
      mirror-1                                   ONLINE       0     0     0
        /var/snap/lxd/common/lxd/disks/2G_1.img  ONLINE       0     0     0
        /var/snap/lxd/common/lxd/disks/2G_2.img  ONLINE       0     0     0

errors: No known data errors
# zpool list test3g
NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
test3g  4.62G   156K  4.62G        -         -     0%     0%  1.00x    ONLINE  -

Now we have a striped mirror of 4.62GB usable.

Let's remove the previous 3G parts.

# zpool remove test3g mirror-0
# zpool status
  pool: default
 state: ONLINE
  scan: scrub repaired 0B in 0 days 00:06:45 with 0 errors on Fri May 21 15:20:30 2021
config:

    NAME                                            STATE     READ WRITE CKSUM
    default                                         ONLINE       0     0     0
      mirror-0                                      ONLINE       0     0     0
        /opt/lxc_image/default.img                  ONLINE       0     0     0
        /var/snap/lxd/common/lxd/disks/default.img  ONLINE       0     0     0

errors: No known data errors

  pool: tank
 state: ONLINE
  scan: scrub repaired 0B in 0 days 00:00:10 with 0 errors on Tue May 18 19:42:56 2021
config:

    NAME                                         STATE     READ WRITE CKSUM
    tank                                         ONLINE       0     0     0
      mirror-0                                   ONLINE       0     0     0
        /var/snap/lxd/common/lxd/disks/tank.img  ONLINE       0     0     0
        /nfs-dual/zfs-images/tank.img            ONLINE       0     0     0

errors: No known data errors

  pool: test3g
 state: ONLINE
  scan: none requested
remove: Removal of vdev 0 copied 102K in 0h0m, completed on Fri May 21 15:33:19 2021
    72 memory used for removed device mappings
config:

    NAME                                         STATE     READ WRITE CKSUM
    test3g                                       ONLINE       0     0     0
      mirror-1                                   ONLINE       0     0     0
        /var/snap/lxd/common/lxd/disks/2G_1.img  ONLINE       0     0     0
        /var/snap/lxd/common/lxd/disks/2G_2.img  ONLINE       0     0     0

errors: No known data errors
# zpool list test3g
NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
test3g  1.88G   142K  1.87G        -         -     0%     0%  1.00x    ONLINE  -

The space is down to 1.87G. Note that this may take long depend on your data size, but it will complete OK. Check the message above:

Removal of vdev 0 copied 102K in 0h0m, completed on Fri May 21 15:33:19 2021

You technically reduced the zpool size from 3G to 2G, without down time, without losing data.

Let's verify the data is still there

# cat /test3g/test.txt
test
#

Yes of course.

Note that only the recent version of zfs supports this. Your mile may vary.

Thanks Best Shilin

Nice work, using zpool add instead of zpool attach. AFAIK zpool add is adding the new disk(s) as RAID-0 (or Just a Bunch Of Disks, total size is added up), while zpool attach is attaching the new disk(s) as RAID-1 (mirror, total size does not change). Then zpool remove a part of the RAID-0 forces the data to flow to the remaining part. Hence resize is archived. FYI if the data size is larger than the remaining disks in the zpool can provide, cannot remove ...: out of space error is prompted. You simply cannot zpool remove it. Credits to ZFS as it is quite foolproof. — midnite, Commented Jun 9, 2022 at 15:55
After zpool remove, how to get rid of the indirect-0? serverfault.com/q/1102923/237756 — midnite, Commented Jun 9, 2022 at 19:28
one major caveat from this approach, from what I have heard on the #zfs IRC channel, is that once you add or remove vdevs around, ZFS adds a feature flag to the pool that makes it incompatible with grub. That might have been fixed in later versions of grub, but that's certainly something to check before doing the move, or at least testing in development first. — anarcat, Commented Oct 15, 2022 at 18:54
I'm getting unrecognized command 'status' and unrecognized command 'add' on zfs-2.1.5-1ubuntu6~22.04.1 which I thought was somewhat recent. So that means I need to update? Which version do you use @Wu-Shilin ? — Cadoiz, Commented Jun 22, 2023 at 6:47
@Cadoiz my guess is you're using zfs status instead of zpool status, the two commands manage different aspects of your ZFS setup — Ben Millwood, Commented Apr 16 at 17:44

KK Patel · Accepted Answer · 2021-08-29 11:35:59Z

2

+50

Once you add your disk to zfs pool, by default zfs allocates whole disk for usage and it formats the disk using an EFI label to contain a single, large slice. This is recommended way.

It is not possible to shrink size of volume if you have allocated whole disk. You can technically force it to shrink by gparted or some other tool but that will cause problems with zfs pool and you will loose your data, so that is not recommended at all.

You can reduce size of particular zfs pool and create new pool from free space. Only way possible to make free some space is you can make disk offline (I hope that you have multiple disks in zfs pool), re-format it and use some space for your desired partition and remaining space allocated to zfs partition for example c150d0. When you add disk back to zfs pool, you need add zfs partition for usage. This is not recommended way but should be counted as workaround. Do it at your own risk.

edited Aug 29, 2021 at 11:35

answered Apr 30, 2020 at 17:23

KK Patel

19.4k14 gold badges58 silver badges81 bronze badges

1

"You can technically force it to shrink by gparted" - you are not forcing it to shrink, you are truncating the pool ("cutting its feet"), so "you may lose your data" is VERY understated: data loss is fairly certain. At least in OpenZFS there is no recommended way of shrinking a partition. You need to create a new pool and migrate the data (send/recv).
– Valmiky Arquissandas
Commented Aug 28, 2021 at 23:26
GParted (1.3.1) does actually not allow shrinking the partition on my system.
– Cadoiz
Commented Jun 22, 2023 at 6:49

Add a comment |

Stack Exchange Network

How can I shrink a zfs volume on ubuntu 18.04?

2 Answers 2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged
18.04
zfs
.

Hot Network Questions

How can I shrink a zfs volume on ubuntu 18.04?

2 Answers 2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged 18.04zfs.

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
18.04
zfs
.