How to recover partially formatted ext4 partition (testdisk did not help)

Question

I have а 2TB "single ext4-formatted partition" HDD. Recently, while working under Windows 7 on the same machine, I have damaged the afore-mentioned partition. What happened is that while I was trying to recover external USB drive I opened the standard Windows Disk Management tool and it prompted me if I want to make disk1 "active". At that moment I didn't realize that "disk1" is not my USB disk, but the ext4 internal disk. After clicking OK, Windows has created a 100MB "system reserved partition" on the disk and left the rest untouched.

When I logged back in to Linux, the partition was not mounted. I deleted the created by Windows 100MB NTFS "system reserved partition" and tried to run Testdisk. Testdisk sees my old ext4 partition, but at the end tells something like "no partition can be recovered".

Is it possible to get back my partition and directory structure + data? The problem is that I had there all my programs (C/C++) and working directories as well as some experimental data, i.e. files which don't have specific headers like jpg or pdf. Is there a way to recover this data? Can I, for instance, recover part of the partition, let's say whatever is above 100MB?

P.S. Please let's avoid "you should have done a backup"-like comments.. :) I had a backup, but it was stored on the USB drive which I was trying to recover first.. (a general disk failure, disk is not readable).

Answer 1

You may have run into this problem, but that's not really clear. If so, yours is the first case of its kind that I've heard of, and you can thank the GNU parted developers for sitting on a fix for over two years.

As a practical matter, your best bet for recovery is this:

1. (Optional, but strongly recommended.) Do a low-level backup of the damaged disk. For instance, sudo dd if=/dev/sdc of=/path/to/backup.img backs up /dev/sdc (if that's the damaged disk) to /path/to/backup.img (you can change this target directory or filename, if you like). Obviously you must have sufficient free space in /path/to to hold the backup. This means you may need to buy a new backup disk, prepare it, and mount it at /path or /path/to. You can skip this step, but then if you make matters worse, it will become harder to recover -- perhaps to the point of becoming impossible to recover.
2. Use fdisk (if the disk used MBR) or gdisk (if the disk used GPT) to re-create your original partitions. Since you say you had a single partition on the disk, chances are that it started at sector 2048 and extended to the end of the disk. Thus, deleting the current partition(s) and re-creating that one may work. Do not create a fresh filesystem on the disk; just create an empty partition. (GParted likes to create a fresh filesystem and so should be avoided. It's possible to get it to create an empty partition, but why take the risk?)
3. Reboot. (This just ensures that you're using the new partition table. There are less radical ways to do this, but this one is easy to describe.)
4. Run e2fsck on the partition you just created. If you're very very lucky, it may locate enough of the ext4fs data structures to bring the disk back to live, although probably with some missing files or other problems. You might want to use fsck.ext4 to make sure, that it recognizes properly that it is a ext4 filesystem. Fiddling with e2fsck options (such as -b) may improve your odds of recovering data, but these options are very advanced. See the e2fsck man page for details.
5. If this fails, run PhotoRec on the disk. This should enable recovery of individual files, although they'll probably lack filenames or any sort of organization, so you'll need to sift through them to figure out what they all are.

Answer 2

1. For recovery I was using only the "dd"-copy of the disk:

   sudo dd if=/dev/sdb of=/dev/sdd
   

("sdb" is the damaged disk and "sdd" is the one I used for tests)

2. First, I used "gdisk" to create a new empty GUID partition table (option -o).

   $ sudo gdisk /dev/sdd
   Command (? for help):   
   $ o
   This option deletes all partitions and creates a new protective MBR.
   Proceed? (Y/N): 
   $ y
   $ w
   

3. I tried sudo e2fsck /dev/sdd - unsuccessful (first superblock was corrupted)

   $ sudo e2fsck /dev/sdd
   e2fsck 1.41.12 (17-May-2010)
   e2fsck: Superblock invalid, trying backup blocks...
   e2fsck: Bad magic number in super-block while trying to open /dev/sdd
   
   The superblock could not be read or does not describe a correct ext2
   filesystem.  If the device is valid and it really contains an ext2
   filesystem (and not swap or ufs or something else), then the superblock
   is corrupt, and you might try running e2fsck with an alternate superblock:
       e2fsck -b 8193 <device>
   

4. then I did sudo mke2fs -n /dev/sdd to list the superblocks

   $ sudo mke2fs -n /dev/sdd1
   mke2fs 1.41.12 (17-May-2010)
   /dev/sdd is entire device, not just one partition!
   Proceed anyway? (y,n) 
   $ y
   
   Filesystem label=
   OS type: Linux
   Block size=4096 (log=2)
   Fragment size=4096 (log=2)
   Stride=0 blocks, Stripe width=0 blocks
   122101760 inodes, 488378646 blocks
   24418932 blocks (5.00%) reserved for the super user
   First data block=0
   Maximum filesystem blocks=0
   14905 block groups
   32768 blocks per group, 32768 fragments per group
   8192 inodes per group
   Superblock backups stored on blocks:
   
           32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
           4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
           102400000, 214990848
   

5. then - sudo e2fsck -b 214990848 /dev/sdd after applying several fixes I ended up with a partially recovered partition! In fact, it was not so straight-forward. I tried with several superblocks and different e2fsck options. Anyway, this was the general recipe.

Addendum: during the recovery I replied "no" to all questions like:

One or more block group descriptor checksums are invalid. Fix(y)?
$ no
/lost+found not found.  Create(y)?
$ no
Free blocks count wrong for group # (32768, counted=0). Fix(y)?
$ no
Free inodes count wrong (122101749, counted=1606). Fix(y)?
$ no

Answer 3

Try GNU ddrescue.

GNU ddrescue is a data recovery tool. It copies data from one file or block device (hard disc, cdrom, etc) to another, trying hard to rescue data in case of read errors.

The basic operation of ddrescue is fully automatic. That is, you don't have to wait for an error, stop the program, read the log, run it in reverse mode, etc.

For more info, and to download ddrescue, see their website.

Ubuntu Rescue Remix is a GNU/Linux live system which runs from CD or USB flash device. It provides the data recovery specialist with a command-line interface environment equipped with the best free-libre, open source data recovery and forensics tools available.

There are also great products, but not free. Such as R-STUDIO for Linux Data Recovery.

You can also try R-Studio.

R-Studio is a family of powerful and cost-effective undelete and data recovery software. Empowered by the new unique data recovery technologies, it is the most comprehensive data recovery solution for recovery files from NTFS, NTFS5, ReFS, FAT12/16/32, exFAT, HFS/HFS+ (Macintosh), Little and Big Endian variants of UFS1/UFS2 (FreeBSD/OpenBSD/NetBSD/Solaris) and Ext2/Ext3/Ext4 FS (Linux) partitions. It also uses raw file recovery (scan for known file types) for heavily damaged or unknown file systems. It functions on local and network disks, even if such partitions are formatted, damaged or deleted. Flexible parameter settings give you absolute control over data recovery.

^{Source:GNU & R-Studio}