Recover from a corrupted filesystem when fsck do not help

Question

Something went wrong with my filesystem, Ubuntu set it to read-only and now under Ubuntu Live Disc, fsck can't fix it.

I'm running 13.04 and it won't boot - at startup, it just shows the grub rescue prompt.

It's a straight-forward setup, just one hard drive on /dev/sda1 but it won't even mount.

The install program can see the partition, that it's ext4 and that it's the boot partition.

However it seems that I can't rescue the filesystem by doing an Ubuntu installation with the Ubuntu live disk because it gives no indication whether it's about to overwrite the whole lot so I don't want to risk it.

I've got a back-up using backuppc but stupidly I lost my rescue disks. I'd rather avoid a full installation followed by a restore which I've no experience at executing.

The crux of the matter is that fsck says it fixes everything but actually doesn't, so the next time I run it, I get exactly the same error messages and fixes.

Here's the output:

ubuntu@ubuntu:~$ sudo fsck.ext4 -vy /dev/sda1
e2fsck 1.42.8 (20-Jun-2013)
ext2fs_check_desc: Corrupt group descriptor: bad block for block bitmap
fsck.ext4: Group descriptors look bad... trying backup blocks...
Block bitmap for group 0 is not in group.  (block 2553887680)
Relocate? yes

Inode table for group 0 is not in group.  (block 2440124416)
WARNING: SEVERE DATA LOSS POSSIBLE.
Relocate? yes

One or more block group descriptor checksums are invalid.  Fix? yes

Group descriptor 0 checksum is 0x761e, should be 0xcf25.  FIXED.
Block bitmap for group 4352 is not in group.  (block 2553887680)
Relocate? yes

Inode table for group 4352 is not in group.  (block 3731970048)
WARNING: SEVERE DATA LOSS POSSIBLE.
Relocate? yes

Group descriptor 4352 checksum is 0x5eda, should be 0x3da3.  FIXED.
Inode bitmap for group 4353 is not in group.  (block 2785042439)
Relocate? yes

Group descriptor 4353 checksum is 0xd8b1, should be 0xedfb.  FIXED.
Inode bitmap for group 4354 is not in group.  (block 838860807)
Relocate? yes

Group descriptor 4354 checksum is 0x1718, should be 0x0438.  FIXED.
Inode bitmap for group 4355 is not in group.  (block 771751943)
Relocate? yes

Group descriptor 4355 checksum is 0x0bc8, should be 0x4170.  FIXED.
fsck.ext4: e2fsck_read_bitmaps: illegal bitmap block(s) for /dev/sda1

/dev/sda1: ***** FILE SYSTEM WAS MODIFIED *****

/dev/sda1: ********** WARNING: Filesystem still has errors **********

ubuntu@ubuntu:~$ 

That is exactly the same as it was 10 times earlier and I'm sure the next ten times I try it - exactly the same checksums and block ids. Any help gladly received!

Thanks.

EDIT: basically I guess the question is: is this filesystem repairable in situ now or does that info from fsck mean my disk is dead? And if it's not dead, what can I do above and beyond what I did with fsck?

EDIT: used tune2fs to identify superblocks and ran e2fsck -b 01234 /dev/sda1 as an alternative to fsck... no effect.

EDIT: trying testdisk which tells me the partition is bad. ... OK testdisk doesn't seem to offer much.

Answer 1

Finally found this link where file system type ext4 gets a bashing but after giving all the tips I'd already tried, it finally says do this:

sudo mkfs.ext4 -S /dev/sda1

This will replace all your superblocks with correct data, assuming the blocksize is guessed correctly (the default is correct for most systems.) If you need to use this, please read the man page on -S first. Don't blame me!

but only if you feel lucky.

It fixed the partition so I could read it again. However, I had to run fsck to fix the errors that were still there, and that dumped the content of /etc and a lot of other stuff into /lost+found so I'm going to have to do a re-install and restore from back-up to get it going again.

Answer 2

First: If you have important data on this disk this would be a good time (actually a bad time) to make a backup. See Data-Recovery: Imaging a damaged device, filesystem or drive. Maybe your Harddisk is dying.

Second: Look at this: How can I fix mounting my data drive after a crash?

Third: Check your Harddrive using Smartmontools and eventually badblocks: sudo badblocks -vsn /dev/sda (This may take some time, don't do this if you have an ssd)