I was trying to take a hard drive out of a large LVM group consisting of 5 drives. It had a 5.5TB filesystem (ext4). The LV was 7.5TB. I wanted to reduce the LV by 1.82TiB so that I then could shrink the VG and pvmove the drive off the group.
I did the following:
fsck -f (passed)
lvreduce -L 1.82T /dev/mapper/my_storage_vg-storage_lv
this produces the warning:
Rounding up size to full physical extent 1.82 TiB WARNING: Reducing active logical volume to 1.82 TiB THIS MAY DESTROY YOUR DATA (filesystem etc.) Do you really want to reduce storage_lv? [y/n]:
Which is what I expected, so I said yes, assuming I had the space to give.
HOWEVER, you may have noticed I forgot the minus sign on the 1.82T, so instead of reducing it by 1.82T, I instead reduced it to 1.82T!!
My next command was an fsck, which failed and I said no to any repairs, immediately seeing the error.
The next command I tried was:
lvextend -l +100%FREE /dev/mapper/my_storage_vg-storage_lv
This restored the size of the LV.
I then tried
fsck- f
but this time the fsck failed once it got to block bitmap differences.
I suspect the lvextend command put the blocks back in, in a different order then they were in when they were removed, essentially scrambling the filesystem.
Does this sound correct, and if so what can I do to unscramble the ext4 file system? Anything at all?
All help greatly appreciated...
Addendum: I did a pvscan
at an earlier point in the process:
PV /dev/sdf1 VG my_storage_vg lvm2 [1.82 TiB / 0 free]
PV /dev/sda1 VG my_storage_vg lvm2 [1.82 TiB / 0 free]
PV /dev/sde1 VG my_storage_vg lvm2 [1.82 TiB / 0 free]
PV /dev/sdc1 VG my_storage_vg lvm2 [1.36 TiB / 0 free]
PV /dev/sdd1 VG my_storage_vg lvm2 [698.63 GiB / 698.63 GiB free]
I suspect this is the order that the drives were added to the vg originally (maybe, not sure). I am considering taking the LV and VG down to 0 size, then adding in the drives one at a time
vgextend /dev/sda1
lvextend sda1
vg extend sde1
lvextend sde1
etc...
Is there a better way to do this?
Better yet:
lvreduce -l 0 /dev/mapper/my_storage_vg-storage_lv
lvextend /dev/mapper/my_storage_vg-storage_lv /dev/sdf1
lvextend /dev/mapper/my_storage_vg-storage_lv /dev/sda1
lvextend /dev/mapper/my_storage_vg-storage_lv /dev/sde1
lvextend /dev/mapper/my_storage_vg-storage_lv /dev/sdc1
fsck -f /dev/mapper/my_storage_vg-storage_lv