Resilience of Btrfs inside RAID5

Question

Btrfs currently lacks fsck, risking corruption if there's a power failure. To what extent, if at all, would creating a btrfs filesystem within a RAID5 array mitigate this?

Answer 1

AFAIK it would make no difference.

RAID is an abstraction layer for physical disks, it makes multiple disks/partitions look like a single "block device" to the file system. Some RAID levels (including RAID1 and RAID5) can handle physical disk failures transparently: Remove a disk, and for the file system it looks like nothing has changed.

But the file system works on top of the "block device" abstraction layer. Using a RAID5 as the block device helps dealing with physical disk failures, but it does nothing for the file system itself, so the risk of file system corruption remains the same.

RAID blocks (mdadm 'chunks' with default size 64kB) differ from file system blocks (ext4 default size 4kB) and are designed to detect a different kind of corruption.

RAID5 works by splitting data from the file system and writing one chunk to each of N-1 disks and a checksum to the Nth disk. RAID5 is designed to detect "corrupt data" in the sense of bad sectors, in which case the original can be restored from the checksum and the N-2 remaining good chunks.

But if the FS updates the metadata and dies before it has time to write the files, the file system may be corrupt without the RAID being any wiser: What was written to disk is the same as what is read back from disk, as evidenced by the checksums matching. The RAID cannot know that important pieces are missing from the perspective of a higher level application.

See this paper (pdf) for examples of corruption that goes undetected by RAID. In the context of file system corruption, I guess "torn writes" (only some of the data that were supposed to be written actually are) is particularly relevant.

Answer 2

Linux software RAID5 would actually make the power failure situation worse - much worse - for the reason already addressed. Even on ext3/4, replaying the journal after a power failure will not fix the RAID5 inconsistency which invariably arises due to the software RAID not having a nonvolatile "journal" of its own to play back. The result is corruption of not only user data but also filesystem data in the event of a power failure during a write operation. Dedicated hardware RAID controllers with battery-backed cache do not suffer from this, but that still doesn't make up for the lack of being able to run fsck on btrfs.

If you care about your data integrity then do not run btrfs without an fsck tool, and do not run RAID5 (or any parity RAID level) without nonvolatile cache or a UPS.