3

I'm looking for advice on partitioning a new system with multiple disks for root on ZFS with 21.04. The machine has:

  • 32 GB ECC RAM
  • Two 5 TB HDDs
  • 1 TB M.2 SSD
  • 2 TB M.2 SSD

For context, I've read Aaron Toponce's ZFS administration guide, Ubuntu 20.04 Root on ZFS and ZSys dataset layout. I have some ZFS experience: currently I have another machine running 20.04.2 LTS (but originally set up with 16.04) with / on an ext4 HDD and /home on ZFS with two mirrored HDDs, plus some other datasets that I manage explicitly.

Here, in brief, is what I tentatively think I want:

  • The two HDDs in a mirrored vdev for rpool.
  • A small partition on each SSD for a mirrored SLOG for rpool.
  • A 2 GB partition on each SSD to be mirrored for bpool.
  • A swap partition.
  • An EFI System Partition.
  • Most of the space on the SSDs used for persistent L2ARC (in a striped configuration).

Questions:

  1. My most fundamental question is whether this approach with L2ARC is a terrible idea. My hope is that—relying, crucially, on the new support for persistent L2ARC—I can get the best of the large HDDs and the fast SSDs without having to manually decide what to put on each kind of storage. Does this make sense? Is there a better way to achieve the same goal? Or, would it be significantly better to just resign myself to making those decisions manually? (In that case, I guess rpool would go on the SSDs and the HDDs would be a third pool.)

  2. How large should my swap partition be? Should I use a single partion, or half the space on each SSD? It's been years since I've used a swap partition rather than a swap file, but I understand it's currently strongly recommended with root on ZFS due to a deadlock bug. Suspend-to-disk isn't critical, but would be nice if it isn't too much of a pain.

  3. I remember reading years ago that using partitions for ZFS (rather than full disks) was not recommended, but it sounds like that's not applicable with SSDs. Do I understand that correctly?

  4. Recommendations for the size of the SLOG? Toponce says, "If you decide to use the same device for both your ZIL and your L2ARC, which is certainly acceptable, you should partition it such that the ZIL takes up very little space, like 512 MB or 1 GB, and give the rest to the pool as a striped (RAID-0) L2ARC." Does that make sense?

  5. I haven't fully digested this caveat from Ubuntu 20.04 Root on ZFS (Step 4, "System Configuration," #8):

    For a mirror or raidz topology, we do not want GRUB writing to the EFI System Partition. This is because we duplicate it at install without a mechanism to update the copies when the GRUB configuration changes (e.g. as the kernel is upgraded). Thus, we keep /boot/grub on the boot pool for the mirror or raidz topologies. This preserves correct mirroring/raidz behavior, at the expense of being able to write to /boot/grub/grubenv and thus the recordfail behavior.

    That seems like a significant downside. Would it make any sense to not mirror bpool, instead? The obvious downside is that it couldn't self-heal from corruption, but, while that would be highly annoying, it wouldn't actually cause data loss: I'd just need to boot from a live usb or something and get back to a bootable state, right?

  6. Broadly, does this make sense overall? Are there things I'm not thinking of? The hardware is what it is, but I'm open to rethinking any and all of the plan for how to use it.

Many thanks!

1
  • 2
    I'm writing this 7 months after your original (excellent) question. As of now there are zero answers and zero comments. I'm thinking of doing something similar and would love to know if you did proceed as you describe above, and if there's anything you would have done differently? Perhaps you could answer your own question ...? Dec 13, 2021 at 17:24

0

You must log in to answer this question.

Browse other questions tagged .