Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I was trying to take a hard drive out of a large LVM group consisting of 5 drives. It had a 5.5TB filesystem (ext4). The LV was 7.5TB. I wanted to reduce the LV by 1.82TiB so that I then could shrink the VG and pvmove the drive off the group.

I did the following:
fsck -f (passed)
lvreduce -L 1.82T /dev/mapper/my_storage_vg-storage_lv

this produces the warning:

Rounding up size to full physical extent 1.82 TiB
  WARNING: Reducing active logical volume to 1.82 TiB
  THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce storage_lv? [y/n]:

Which is what I expected, so I said yes, assuming I had the space to give.

HOWEVER, you may have noticed I forgot the minus sign on the 1.82T, so instead of reducing it by 1.82T, I instead reduced it to 1.82T!!
My next command was an fsck, which failed and I said no to any repairs, immediately seeing the error.
The next command I tried was:
lvextend -l +100%FREE /dev/mapper/my_storage_vg-storage_lv

This restored the size of the LV.

I then tried
fsck- f
but this time the fsck failed once it got to block bitmap differences.

I suspect the lvextend command put the blocks back in, in a different order then they were in when they were removed, essentially scrambling the filesystem.

Does this sound correct, and if so what can I do to unscramble the ext4 file system? Anything at all?

All help greatly appreciated...


Addendum: I did a pvscan at an earlier point in the process:

PV /dev/sdf1   VG my_storage_vg   lvm2 [1.82 TiB / 0    free]
PV /dev/sda1   VG my_storage_vg   lvm2 [1.82 TiB / 0    free]
PV /dev/sde1   VG my_storage_vg   lvm2 [1.82 TiB / 0    free]
PV /dev/sdc1   VG my_storage_vg   lvm2 [1.36 TiB / 0    free]
PV /dev/sdd1   VG my_storage_vg   lvm2 [698.63 GiB / 698.63 GiB free]

I suspect this is the order that the drives were added to the vg originally (maybe, not sure). I am considering taking the LV and VG down to 0 size, then adding in the drives one at a time

vgextend /dev/sda1
lvextend sda1
vg extend sde1
lvextend sde1
etc...

Is there a better way to do this?


Better yet:

lvreduce -l 0 /dev/mapper/my_storage_vg-storage_lv
lvextend /dev/mapper/my_storage_vg-storage_lv /dev/sdf1
lvextend /dev/mapper/my_storage_vg-storage_lv /dev/sda1
lvextend /dev/mapper/my_storage_vg-storage_lv /dev/sde1
lvextend /dev/mapper/my_storage_vg-storage_lv /dev/sdc1
fsck -f /dev/mapper/my_storage_vg-storage_lv
share|improve this question

1 Answer 1

You can't extend the LV and hope to hit the right layout at random. Use vgcfgrestore (I'll let you read the manual) to restore the configuration from before your lvreduce. Do not mount or fsck anything.

share|improve this answer
    
Thank you so much, Gabriel. This is what I needed. However, it did not work entirely. I restored it to the point prior to the lvreduce, but an fsck still showed errors. The only command I invoked on the vg in it's reduced state was an fsck -f. I answered no to a few <fix?> prompts, then ctrl-c the process. Assuming the vgcfgrestore worked, that would imply that the fsck modified things (which I thought required user confirmation). At least I am a step closer.... –  Peter Sep 20 '13 at 15:20
    
I am looking at one of 2 options now I guess: 1) figure out how to undo the fsck changes manually. 2) Allow fsck to fix the system and hope that it can figure out how to undo its changes itself. :/ –  Peter Sep 20 '13 at 15:50
    
You can do lvchange --permission r to make the volume read-only, then use e2fsck -n or mount -o ro to get an idea of the filesystem state. –  Gabriel Sep 20 '13 at 16:14
    
Well, the damage is already done. I know that on the recent fsck. Now I need to undo what fsck did when the LV was small. I'm hoping a fsck -y will fix this. (Ideally I'd make a copy of the LV first, just don't have access to 8TB to do that with... ) –  Peter Sep 20 '13 at 23:19
1  
Success! Filesystem restored. Even had the nerve to try the lvreduce again. This time correctly with lots of read only step checking. Sitting mid pvmove now. :) –  Peter Sep 21 '13 at 5:23

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.