Partition recovery: How can I insert existing partitions into an extended partition?

Question

In trying to solve a problem with Windows, a tool I was using deleted some of my partitions. I used TestDisk to recover my partitions.

While it managed to recover all my data partitions, it couldn't recover my swap partition. Normally, that wouldn't be a big deal, since it's easy to create another swap partition. In this case, though, it also didn't find my extended partition, so all my recovered partitions are primary partitions, instead of logical partitions. Now, I can't create any more.

I want to re-create the layout shown in my pre-event fstab:

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    nodev,noexec,nosuid 0       0
# / was on /dev/sda5 during installation
UUID=65340dbf-705f-409d-8bfa-54f408fe23c9 /               ext4    errors=remount-ro 0       1
# /extra was on /dev/sda8 during installation
UUID=6921d03e-e64f-4403-9381-be2b2eed62e2 /extra          ext4    defaults        0       2
# /home was on /dev/sda6 during installation
UUID=b1c33077-6a9b-4e1c-bde5-d44efc96ffbd /home           ext3    defaults        0       2
# /windows was on /dev/sda1 during installation
UUID=F624495B2449204B /windows        ntfs    defaults,umask=007,gid=46 0       0
# swap was on /dev/sda2 during installation
#UUID=1e3afc3a-152e-44ab-9275-c5ca30efad0f none            swap    sw              0       0
# swap was on /dev/sda7 during installation
#UUID=2e41ce67-2a25-4ed3-a64e-b5480172477b none            swap    sw              0       0
/dev/mapper/cryptswap1 none swap sw 0 0
/dev/mapper/cryptswap2 none swap sw 0 0

# tmpfile stuff
tmpfs /tmp tmpfs defaults,size=512M,mode=1777 0 0
tmpfs /var/tmp tmpfs defaults,size=32M,mode=1777 0 0

Here's my current layout, expressed both as fdisk output and a GParted screen shot:

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000bfc8b

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048    41945087    20971520    7  HPFS/NTFS/exFAT
/dev/sda2        41955328    83898367    20971520   83  Linux
/dev/sda3        83900416   764268543   340184064   83  Linux
/dev/sda4       764276373   970486649   103105138+  83  Linux

GParted screenshot http://img7.imagebanana.com/img/vqt7wwq6/Screenshotfrom2012052800_04_35.png

Clearly, I don't want to delete any data to do this. Are there any tools that are capable of this? It seems that if I could write the partition table at a low level, I'd be able to accomplish this. What do you recommend?

By the way, I use an encrypted home directory (that's $HOME, not /home), and previously my swap was encrypted. I don't know how encrypted swap is set up.

Answer 1

Rather than manually editing the partition tables you could just recreate your old partition table using gparted and either use cp -p or dd to copy the old partitions into the new ones.

External storage requirements

How you do so depends on your external storage capabilities. If you have another drive that you can copy the partitions you should copy the primary partition(s) to an image (for dd) or directory (for cp) on the external drive, then create an empty logical partition and and copy the image into this new empty partition. This would be the best way to go about things.

To back partitions up as images somewhere (sda1 being the first partition on your first hard drive):

sudo dd if=/dev/sda1 of=[path to where you want to store the image] bs=1M noerror

To copy partitions to a directory (preserving permissions) first mount them then:

sudo mkdir /[path to where you want to store them]/partition-backup1
sudo cp -pr /media/mounted-partition-to-be-copied /[path to where you want to store them]/partition-backup1

See below for more information on cp and dd.

If you don't have external storage

However, if you don't have enough space/time to copy to partitions to external storage first, you could potentially use creative resizing/copying to do the entire move on the same drive. In other words, you could resize partitions with a lot of free space down to a smaller size give you the room to create new duplicate logical partitions and then delete the old one and resize back up after the copy. You would also need to temporarily store one partition as an image inside of another one so you could create the extended partition.

However, be careful if any of the partitions (/home, swap, etc) are encrypted. If so it would be very, very dangerous to resize them. There is a guide for resizing encrypted partitions here but I wouldn't recommend doing so unless you really cannot find any external storage to avoid this step.

Copying the old primary partitions into new logical partitions

First make sure you have a liveCD/USB handy, you will probably need to reinstall GRUB later.

I'm assuming here that you have assessed the method best for you and have now created an empty new logical partition and have access to either the old primary partition or an image of the partition stored externally.

You can either use cp which copies file by file or dd which copies blocks of raw data. For any encrypted partitions you will have to use dd. I personally feel that when used correctly dd is safer because you don't have to worry about permissions/ownership/symlinks/etc. However if you make a mistake using dd you will break something.

To use cp: (for non-encrypted partitions) first mount both the old primary version of the partition and the new empty logical partition. The copy the files from the old partition to the new ones. The -p option preserves the permissions and owners of the files so you don't mess up your permissions, -r just copies recursively. However it cannot create files with owners above the permissions of whoever is running the command so you shoul run this command as root.

sudo cp -pr /media/[mounted-old-primary]/ /media/[mounted-empty-logical]/

This might take awhile.

To use dd: You need to be absolutely sure of the partition names of your old and new partitions. You should also be cloning partitions that are exactly the same size (or to a partition that is larger--at your own risk). Do not try cloning to a partition that is smaller than the original. dd will overwrite any device it is ordered to copy to. Either use the gparted GUI to get these right or you can run sudo fdisk -l to see the partition table. Once you are sure of your names run the dd command (must be run as root to write directory to devices)

People googling in: DO NOT run the following commands unless you understand what they do and have changed the partition names to the partitions your want to copy/overwrite

sudo dd if=/dev/sda2 of=/dev/sda6 bs=1M noerror

In this example sda2 is the original primary partition and sda6 is the new, empty logical partition. Be sure to check that you've changed the pathes correctly before each operations.

If you are copying from an image you backed up, run a command like this:

sudo dd if=/path/to/image.img of=/dev/sda6 bs=1M noerror

Either of these will also take awhile. You can make sure they are still running by checking top and noting various related processes chugging along.

Cleaning up

Do not delete anything until you are sure that it has copied correctly and that you will not need the backup anymore. That said once you've successfully copied everything onto the new logical partitions you will want to remove the old primary partitions and resize all the partitions to how you want them.

You haven't touched the MBR at all but if you've moved the partition with /boot on it, you will need to reinstall/upgrade GRUB so it points to the new partition.

Reinstall GRUB from a liveCD by mounting your / directory (replace sda3 with whatever partition it is for you)

sudo mount /dev/sda3 /mnt

At this point you could run sudo grub-install --root-directory=/mnt /dev/sda to reinstall grub but you would still need to run update-grub from the native installtion (meaning a rescue boot from the GRUB command line) to update the config files. We can combine both these steps and use chroot to make the / partition the liveCD's / filesystem.

(This part heavily based on https://help.ubuntu.com/community/Grub2/Installing)

Mount the critical virtual filesystems. Run the following as a single command: for i in /dev /dev/pts /proc /sys; do sudo mount -B $i /mnt$i; done

Chroot into your normal system device: sudo chroot /mnt

Reinstall GRUB 2:

grub-install /dev/sda

Recreate the GRUB 2 menu file (grub.cfg)

update-grub

Exit chroot: CTRL+D

Reboot.

sudo reboot

Answer 2

Having and encrypted /dev/sda3 doesn't help matters here. There is no tool that will move partitions onto an extended partition automatically. You are going to have to do this manually, however the good news is that you've only got 70 odd Gb of data on home and extra to back up and restore.

I suggest you back up the data from /dev/sda3 (home) and /dev/sda4 (extra) and delete the partitions. I note that you actually had two swap partitions originally.

Once you've backed up the data, delete /dev/sda3 and /dev/sda4, and in their place make one extended partition. You can then create partitions for /home, swap, and extra as well if you want. I recommend ext4 for /home (not ext3 as you have currently) and not encrypted unless you have really sensitive data on there. consider if "extra" is superfluous to your needs.

Using volume label as a basis for mounting partitions in fstab seems to work very well. It's just a matter of changing 'UUID=uuid' to 'LABEL=volume_label' in fstab once the partition labels have been assigned. This is easier to do manually and easy to change later, since you don't have to deal with spaghetti uuid numbers. I'm not sure you can assign a volume name to the swap partition however.

Chris