4

I want to ask wich file system is the most damage and file corruption resistant. I mean the most secure for my data preventing data loss and such in case of a blackout or power failure.

For example: I want my data to be safe if someone disconnects my Ubuntu machine from the power outlet by accident.

2
  • 1
    I think this a valid question and not too broad for AU. The answers show that this can be answered in a meaningful manner that will help future users decide what filesystem to deploy on their PC. Apr 4, 2014 at 19:16
  • 1
    If there is a hardware related problem your mobo getting fried or the connectors of the hdd getting fried is not going to help if your filesystem is resistant to file corruption. To prevent problems you create BACKUPS regularly and also check those backups regularly.
    – Rinzwind
    Apr 4, 2014 at 19:38

2 Answers 2

3

There are various levels of corruption resistance in file systems in the event of loss of power, from least to most corruption-resistant.

  1. File systems with no journalling.

    Examples: ext2, fat32, fat16

    Such file systems have no protection against file system corruption due to loss of power. After a power loss during a write, the metadata for the file may be in an inconsistent state, meaning that the file system is effectively corrupted and needs repair. A repair won't necessarily be able to successfully recreate a file's metadata from a single point in time. In some cases, for example, a repair tool may not be able to know certain details about a file.

  2. File systems with a writeback journalling mode.

    Examples: ext3/ext4 with data=writeback, NTFS

    Writes to the metadata for files is journalled. After a power loss during a write, the metadata for the file can be rolled back leaving the file system in a consistent state requiring no repair. However, any changes to the actual data of a file won't be able to be rolled back and the file may be left in an inconsistent state.

    A file may even be left with old data from before a write operation even if its metadata is successfully committed.

  3. File systems with an ordered journalling mode.

    Examples: ext3/ext4 with data=ordered

    Pretty similar to the writeback mode. Writes to the metadata for files is journalled, and the system ensures that the file data is written before the metadata change is committed in the journal. After power loss during a write, the metadata can be rolled back leaving the file system in a consistent state requiring no repair. Any changes to the actual data of a file won't be able to be rolled back and the file may be left in an inconsistent state.

    Where this differs from the writeback mode is that you at least will never be left with older data in a file than its metadata, and will never be left with data that previously didn't belong to that file. If metadata is committed, then it is guaranteed that the file data is at least up to date with the metadata, though it may still contain some newer data from part of a later write to that file.

    Having this constraint can improve security and improves the ability for applications to implement their own data integrity measures that rely on the metadata for the file at least reflecting a write that was made to the file at some stage.

  4. File systems with full journalled mode.

    Examples: ext3/ext4 with data=journal, Reiser4

    Writes to both the metadata and all file data are both journalled. After power loss during a write, both the file's metadata and all its data can be rolled back to the state it was in prior to the failure.

    This mode has the highest performance penalty because all data must be written twice, effectively cutting throughput in half. Thus, it is rarely used. Applications that rely on data integrity can take their own measures to ensure file data can be recovered from an inconsistent state, as long as they can trust the metadata on files.

  5. Special mention: log-structured file systems or copy-on-write

    Examples: btrfs, ZFS

    Some file systems such as btrfs offer similar protection to a full journalled file system without as much performance penalty. btrfs is a log-structured file system. Writes to both the metadata and all file data are written to a big long write log which allows both file metadata and contents to be rolled back completely, but this requires no separate "commit" step for the file data, as the file data is duplicated as it's written to, allowing for rollback without writing twice. If a write fails due to a power loss, the log is only replayed to the last successful write prior, which will still provide access to the old version of the file in its fully consistent state.

    ZFS is a copy-on-write filesystem which has similar benefits to a log-structured file system.


Note: Write caching and Barriers

Regardless of the file system, write caching on the drive itself may optimise performance by performing queued write commands in a different order to their place in the queue. However, this would effectively render all journalling useless, or at least all journalling modes other than writeback which is unlikely to be affected much by this optimization.

To combat this however, drives have implemented a concept of "write barriers" where the computer can tell the drive that it must complete all previous writes before a particular write can begin. Modern operating systems use write barriers to ensure that journalling file systems benefit properly from the integrity offered by journalling. In typical file write operations write barriers lose most of the performance benefit you would have gained from the drive's ability to perform writes out of order, but certain applications such as some database servers may still benefit from this even with write barriers enabled.

Write barriers can be disabled in the mount options for a partition but you probably don't want to do this if you are using one of the better journalling modes such as ordered or full journalling.

Some disk drives are designed with their own dedicated back-up-battery which can be used to finish any writes in the write queue in the event of a power loss. If you have such a drive and trust the back-up battery to do this, you can safely disable write barriers and benefit more from out-of-order writes in the drive.


The current default for Ubuntu is ext4 which has an ordered journalling mode. This is much better than no journalling at all, because the file system itself won't be left in a corrupted state, and it's a step up over writeback, because it guarantees that if a file's metadata is updated, its contents have been updated. Note that the default for ext3 was changed to writeback a few years back.

btrfs was designed as a replacement for ext3/4, and future Linux distributions may switch to it as a default, though that is not a certainty. If this does happen, it will bring better ability to cleanly rollback with no significant performance penalty.

4
  • 1
    ext4 defaults to ordered, not writeback, and data journaling does not roll anything back; it just makes sure that any data that hit the journal before the crash is copied to the file. As I mentioned in the other comment, this does not stop you from having a partially written, and thus corrupt file when you crash in the middle of writing to it.
    – psusi
    Apr 2, 2014 at 14:49
  • I believe the default for ext4 was changed from ordered to writeback a few versions of Ubuntu ago - am I incorrect in this belief? You are correct in that a file can always be partially written after a crash, but it's important to distinguish just how "corrupt" it can be - with any type of journalling, the filesystem will not be "corrupt" in the sense of needing filesystem repair, it'll just be that the data in the file is partially written - the ways in which it can be partially written are what varies with the journalling method. Apr 4, 2014 at 23:16
  • Ext3 was made to default to data=writeback instead of data=ordered in Linux 2.6.30. I guess I assumed ext4 was the same but documentation suggests it isn't. Apr 4, 2014 at 23:34
  • ext3 changed the default to ordered in 2.6.36, and all distros had already made sure to use that by default already. See commit aa32a7963.
    – psusi
    Apr 5, 2014 at 2:36
1

The default filesystem, ext4, will be just fine in the event of a power loss, in terms of the filesystem itself, as is any journaling filesystem. If you were thinking about individual files that were not completely saved at the time of the failure, then no filesystem can help those.

9
  • Full journalled file systems (as well as log-structured, and copy-on-write file systems) can actually ensure the integrity of individual files, successfully rolling them back to their state prior to any failed partial write. ext4 can do this in data=journal mode but that comes with a huge performance penalty. btrfs and ZFS could do it with less performance penalty. Apr 2, 2014 at 3:50
  • @neon_overload, no, they can't. They can only ensure that any data that hit the journal before the crash will be copied to the file. Any data that hasn't hit the journal is lost, which may mean you have a partially written, and thus corrupt file.
    – psusi
    Apr 2, 2014 at 14:40
  • Not true. You won't have a partially written, OR corrupt file. The whole purpose of journalling the file data is so that any write operation is atomic - it will either fully complete, or it will be fully rolled back to its state before the write began. It will never be partially written or corrupted. This is of course only possible with a full journalling (or copy-on-write/log-structured) file system which journals the file data. It's not possible if only the metadata is journalled (like default ext3/4). In that case you can have the file data left in a partially written state. Apr 2, 2014 at 23:37
  • 1
    @neon_overload, you are wrong. write() calls are never atomic. More importantly, applications almost never write() the entire file in one go. See Ted Tso ( the author of ext[234] ) correcting someone else with this incorrect belief recently here: bugzilla.kernel.org/show_bug.cgi?id=70121#c7
    – psusi
    Apr 3, 2014 at 0:00
  • AFAIU from that, his point is in reference to an application that is incrementally writing to a file, such as by holding a file open and using multiple calls to write(). Full journalling should still ensure any single write operation is atomic but when an application holds a file open and submits multiple write operations then the file system could be interrupted in between subsequent writes. There is no way for an application to open a transaction to cover multiple writes. Now, do you claim that a single write() is not atomic under full journalling? If not, why would full journalling exist? Apr 3, 2014 at 0:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .