Since upgrading to 16.04 I've moved my /home to a zfs-pool to familiarise myself with zfs.
Already showing its strength it has identified the poor quality of my long running drives:
# zpool status -v pool: homepool state: DEGRADED status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Sun Oct 16 11:09:17 2016 402G scanned out of 1,63T at 76,8M/s, 4h41m to go 401G resilvered, 24,10% done config: NAME STATE READ WRITE CKSUM homepool DEGRADED 0 0 0 mirror-0 DEGRADED 0 0 0 sdb UNAVAIL 0 0 0 sdc ONLINE 0 0 0 sdd ONLINE 0 0 1K (resilvering) mirror-1 ONLINE 0 0 0 sde ONLINE 0 0 0 sdf ONLINE 0 0 0 cache sda4 REMOVED 0 0 0 errors: No known data errors
sdb has been chugging along for years, but with zfs checksumming it has become apparent that it sometimes produces bad data. Such behaviour is of course not accepted by zfs, which has marked the drive as faulty - which led to me adding
sdd thus initialising the resilvering.
Now - zfs has removed
sdb from the pool twice. Both times
sda4 has been removed as well.
sda1 contains my
/. On both occasions it has been found in a
ro-state subsequent to the removal of
Is this all connected, or is it more likely I have more than one faulty drive?
My reasoning is that
sdb is broken, but since it is cached some of the errors might be seen as coming from the cache and since the cache is on the same physical drive as the
/-partition the root gets nuked when
zfs tries to recover from errors seen on
sda4 due to both physical drives being marked as garbage producers.
Does that sound likely?
Trying to remount
/ does not work:
root@computer:~# df / -su: /bin/df: Input/output error root@computer:~# df /de-su: cannot create temp file for here-document: Read-only file system -su: cannot create temp file for here-document: Read-only file system ^C root@computer:~# mount | grep sda /dev/sda1 on / type ext4 (ro,relatime,errors=remount-ro,data=ordered) root@computer:~# mount -o remount,rw /dev/sda1 / mount: cannot remount /dev/sda1 read-write, is write-protected
Since I added the extra disk (thus removing the sata cd-rom drive) I've not seen the problem.
# zpool status -v pool: homepool state: ONLINE scan: scrub repaired 0 in 6h37m with 0 errors on Wed Oct 19 15:23:30 2016 config: NAME STATE READ WRITE CKSUM homepool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 sdb ONLINE 0 0 0 sdc ONLINE 0 0 0 sdd ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 sde ONLINE 0 0 0 sdf ONLINE 0 0 0 cache sda4 ONLINE 0 0 0 errors: No known data errors
All has seemingly been good for almost two weeks, but the broken disk is still there, I'd imagine it should've kept on spewing out garbage data, but it does not seem to.