1

I had to copy disk image from file to external card. Unfortunately Ubuntu switched disks during reboot - sda was laptop hdd, sdb was card; after reboot sda was card, and sdb was hdd. I did dd if=image.dd of=/dev/sdb, as last time it was correct. Image size was 4 GB, so I suppose that I overwrote first 4 GB of my hdd. Image contains minimal ubuntu.

Now I'm stuck with unresponsive system. Laptop is turned on, terminal is up, I had also nautilus opened, but it crashed; before that I checked that my data in /home is untouched (at least I hope it's untouched, but my files are there). How many data I lost? Only OS?

I don't have Ubuntu image right now. I need to recover data from /home. What can I do right now? What should I avoid - I know I shouldn't turn laptop off or reboot, but can I suspend it? Can I logout (lock)?

I can take laptop home and install Ubuntu again, but I need these /home files. Can I rebuild partition table somehow? I have partition info before and after accident too.

  • 3
    Whatever you do, DO NOT continue using your computer! The longer it's running the more chance of messing it up further! What I did when I had a similar situation is I imaged the corrupted HDD to an external one and then attempted recovery on the external. That way, if you screw anything up, you can always re-image it from the original. – Android Dev Sep 8 '17 at 11:49
  • @AndroidDev This ought to be an answer in itself. – Zeiss Ikon Sep 8 '17 at 11:50
  • 2
    @AndroidDev, There is actually an advantage to leaving the computer running, in that you can recover partition start and end points from the kernel. See my answer for details. That said, the risk you describe is also real. Personally, I'd leave it running, at least long enough to extract the partition data from the kernel. – Rod Smith Sep 8 '17 at 14:18
  • 1
    @RodSmith True, but OP stated "Now I'm stuck with unresponsive system" and if he overwrote the first 4GB chances are much of the Linux system is history if it was installed onto the first partition of the drive. – Android Dev Sep 8 '17 at 14:28
  • I missed that detail. Keeping it running becomes a moot point in this case, then. – Rod Smith Sep 8 '17 at 14:33
3

First, understand the difference between partitions and filesystems. In this answer, I will use these terms very precisely, so I want to be sure you understand the distinction:

  • A partition describes a set of sectors on your hard disk. That is, a partition can be sectors 2048 to 1126439 or sectors 4198440 to 15628053134. Partitions are defined in partition tables, which are simple data structures that contain those sector numbers and a few other pieces of partition metadata, such as type codes and possibly names. Two common partition table types are the Master Boot Record (MBR) and the GUID Partition Table (GPT). The former was the dominant partition table type until about four or five years ago; but with the introduction of Windows 8 and the rapid shift from BIOS-based to EFI-based computers, GPT has risen dramatically in popularity. Which partition table you used is critical one for your question, as described shortly.
  • A filesystem is a much more complex data structure that typically resides within a partition (or sometimes another container, such as a logical volume in an LVM setup, or even a file, as in a downloaded .iso image file). Filesystems enable the computer to locate, read, write, and otherwise manipulate individual files. Common filesystems on Ubuntu include ext2/3/4fs, Btrfs, XFS, and JFS. Some others, like FAT, are often used for cross-platform purposes. NTFS is the native filesystem for Windows, and HFS+ for macOS (although Apple is transitioning to APFS). If your computer boots in EFI mode, chances are the first partition held a FAT filesystem; but that filesystem would also have been small and would therefore be completely overwritten and will not be recoverable. (Fortunately, re-creating this EFI System Partition, as it's called, is possible.)

The terms partition and filesystem are sometimes used interchangeably, but the distinction is important for your case. Ultimately, you're most interested in the filesystem data, but to access a filesystem, its starting point must be known, and that start point is normally provided by the partition data.

Given your current situation, there are several possible ways to recover your partition table and/or locate the start point(s) of at least some partitions:

  • Use GPT backup data -- Both GPT and MBR store the partition table data at the start of the disk, which your accident has erased; but GPT also stores a backup of this data at the end of the disk. Thus, you should be able to use the GPT backup to recover the entire partition table. (Note that this will not recover the rest of the data you've overwritten; it's likely that at least one filesystem will be damaged.) AFAIK, the best Linux tool for this type of recovery is my own gdisk, which comes standard with Ubuntu. See this page for details. In brief, you'll launch gdisk on the disk and, if it asks you, tell it to use GPT. gdisk might recover everything at that point, but you may need to type r to enter the recovery & transformation menu and then use b and/or c to read the backup metadata and partition table, respectively. You can use p to view the partition table as gdisk currently sees it. When you're convinced it's recovered, type w to save the changes. Be careful, though! Once you type w, gdisk will overwrite both the main and backup GPT data, so if you do this before you recover the correct partitions, you won't get a second chance at recovery! If you're uncertain, quit out of the program without saving changes by typing q. You can then do more research and try again.
  • Use Linux's in-memory partition data -- So long as you do not reboot or use certain commands related to partitions, the Linux kernel maintains a list of partitions, including their start points and sizes. You can read this information from the /sys/block/{disk}/{part}/ directory in its start and size files. For instance, /sys/block/sda/sda1/start has the start point of /dev/sda1 and /sys/block/sda/sda1/size has the size of /dev/sda1. You can use this information to re-create a partition table. You'd extract the information on all your partitions (say, cat /sys/block/sda/sda*/start and cat /sys/block/sda/sda*/size to get the start points and sizes of all partitions on /dev/sda), then use fdisk, gdisk, or parted to re-create partitions that match what the kernel has recorded. (Caution: Some partitioning tools, such as GParted, create new filesystems when new partitions are created. It's imperative that you not do this; you want to create new partitions without modifying a single byte of the contents of the partitions you create.) Note that you'll still need to fill out metadata that the kernel does not record, such as partition type codes. Also note that the kernel records the start point and size (length) of each partition, but many partitioning tools ask for the start point and end point, the latter being the start point plus size minus 1. Be sure to understand what your partitioning tool wants and feed it the correct values.
  • Use TestDisk -- If your computer uses MBR and you've rebooted, or if the preceding approaches fail, you can use TestDisk to recover partitions that match your filesystems. TestDisk works by scanning the entire disk for filesystem data structures. It can then modify your partition table, or create a new partition table, with partitions that match the filesystems it finds. Note that TestDisk is likely to fail for your first partition or two; however, this approach may work for recovering partitions that begin beyond the end of the area you've accidentally overwritten.
  • Do it blindly -- The first partition on the disk usually begins at a fairly predictable location. In particular, sector 2048 is a common starting point for disks partitioned recently (in the last four or five years). Older disks usually had a first partition that began on sector 63. Some disks, particularly those partitioned in macOS with GPT, have a first partition that begins on sector 40. Thus, if you know enough, you might be able to correctly guess where the first partition began. Given that you've wiped the first 4GB of your disk, though, this knowledge may be worthless; many disks have small (1MiB to 1GiB) first partitions, so you may well have completely overwritten the entire first partition, and well into the second one. If you know that the first partition was a specific size that was much bigger than this, though, or if it filled the entire disk, knowing the start point may enable filesystem-level recovery tools to restore some of the filesystem's data; or you may then be able to recover an intact second partition. This approach is obviously one that's very risky, but if you're desperate enough, you might get lucky with it.

Overall, your best bet is if you used GPT; in that case, your backup GPT data should be intact and you should be able to recover the entire partition table. Even in this case, though, the filesystem data may be partially or completely wiped, at least for the first partition or two. Recovering any data in the first 4GB (the area you accidentally wiped) will be impossible. If a partition begins within that wiped area but extends significantly beyond it, you might be able to use fsck (or similar tools in other OSes) to recover the filesystem, or at least most of it. There are advanced fsck options that might help you with this task, but I'm not an expert on their use. Note also that fsck is really a front-end to filesystem-specific tools such as e2fsck, and filesystem-specific options for these tools may be important in recovering your filesystem(s).

If you haven't rebooted, you might try backing up whatever filesystem(s) are most at risk, using ordinary file-level commands such as tar or cp. There is a risk that Linux will attempt to read filesystem data from the overwritten section of the disk, become confused, and completely hang the computer. In a worst-case scenario, the confusion might even cause writes of corrupted data beyond the damaged area, thus making matters worse. Thus, this approach is not without its risks. Even so, it might be worth taking the risk to back up important user data to an external drive.

Another data-recovery approach is to use PhotoRec. This tool searches the disk, much like TestDisk, but PhotoRec searches for common file types. Thus, it can be used to recover files even if the filesystem data structures are badly damaged. This tool might be useful if a partition began in the damaged first 4GB of your disk but extends beyond that area. If fsck can't recover the filesystem well enough to be mounted, PhotoRec might at least recover some files from it. Be aware, though, that PhotoRec does a poor job of recovering filenames and directory structures. You'll be left with a mound of poorly-named files you'll have to search manually to figure out what you've got.

If you booted in EFI mode, your ESP is almost certainly gone. If you can recover the partition table, you can create a fresh FAT filesystem on the ESP and then use Boot Repair to return Ubuntu to bootability. Conceptually similar repair procedures are possible for Windows, but I'm not familiar with them. If you dual-booted with Windows, you should ask about this on a Windows forum. I recommend restoring the Windows boot loader first and then using Boot Repair in Ubuntu. If you do it the other way around, the computer will end up booting straight to Windows. This issue can be fixed, but it's better not to have it at all.

If you booted in BIOS mode, your boot loader is also destroyed. Boot Repair can recover GRUB once your Ubuntu partition(s) are restored. Depending on the location of Windows (if you dual-booted it), you may need to recover its boot loader using Windows techniques and tools.

  • I have fdisk -l info I gathered before dd: i.imgur.com/LVsxZ4j.png - I think it changes things a little – Bielecki Sep 10 '17 at 16:04
  • No, the fdisk output you showed changes nothing. That's just the MBR partition table from the disk image that overwrote the start of the disk. You could use that to access the disk image, but that's not the data you want to recover. – Rod Smith Sep 11 '17 at 17:46
1

Assuming (when you can boot, say from a Live media) that you find the /home partition is the same size it was before, you ought to be able to recreate the / partition in the same space it occupied before, reinstall Ubuntu with the "save /home folder" option, reinstall your added packages, and pick up where you left off.

As extra insurance, you might wish to install with a different user name, which will create a new user folder in /home. Once done, you can copy the unhidden files and folders from the old user folder to the new.

If the partition table is actually overwritten, you've got a harder job ahead -- you'll need to use file recovery software before you reinstall Ubuntu to recover the files from the former user folder. The data should still be intact, but the "road map" to reading it has been lost, and you'll have to recreate that map. That's an iffy prospect under the best of conditions, but the first rule is: shut off the computer and leave it off until you're ready to reboot from Live media (or remove the HDD and connect it to another computer) to start the recovery process. With your problem, it's unlikely the running computer will overwrite anything remaining on the drive -- but if it's powered off, it can't.

Quoting from comments:

Usually every time I install unix system I'm creating separate partition for /home/ - i can't remember why I didn't do it this time.

Given you installed this system without a separate /home, whether your personal data is mixed in with the OS files depends strongly on how many times you've updated Ubuntu since you installed it. Each time a file in the OS is replaced, the new file is written in free space before the old one is deleted. Then other files can be written into the space freed by deleting the old OS file. That means your files will mix with the OS more and more the longer you keep the install.

Given the situation as I now understand it, I'd suggest you need forensic recovery tools to have any chance of recovering files that may be mixed in with system files -- and those are generally beyond average users.

  • Remember that I overwrote with another Ubuntu image with own /home and user. As long as I hold OS turned on, I'm pretty sure that I have some access to that data - I see it, so it's there. I'm holding it booted like I'd rm -rf / - I have read that turning it off is the worst I could do in that situation. I shouldn't be able to overwrite anything, because loaded OS thinks that my data are there, so it's not marked as free space... yet. – Bielecki Sep 8 '17 at 11:54
  • 1
    Problem is, if you're overwritten the partition table, whatever stub of the OS is still running (just what was in memory during the dd operation, most likely) won't be able to tell there's anything present beyond the 4 GB of the image; that space will look like unallocated if the partition table is read. The only reason you can see that data is because nothing has reread the partition table and directory (trying to do so is likely what crashed Nautilus). There is nothing you can do with an unresponsive system to recover, at this point, other than reboot from Live media or dismount the HDD. – Zeiss Ikon Sep 8 '17 at 12:06
  • 1
    @Bielecki, I agree with the advice of Zeiss Ikon and Android Dev (in the comment). In addition to that, for the future I recommend that you 1. Backup everything important regularly; 2. Use tools with a final checkpoint, to help you avoid the dangerous dd, nicknamed 'Data Destroyer'. mkusb wraps a safety belt around 'dd`, help.ubuntu.com/community/mkusb – sudodus Sep 8 '17 at 12:13
  • 1
    @Bielecki, It is hard to know where the different files are located in a partition. If there was a separate home partition, it would probably be behind the 4GB limit, but if /home (the home directory) was in the root directory (/), we can only hope that most of your personal data (documents, pictures ...) happen to be be stored in the part of the drive behind 4GB. Anyway, it is worthwhile to try and recover what can be found. – sudodus Sep 8 '17 at 12:49
  • 1
    Given you installed this system without a separate /home, whether your personal data is mixed in with the OS files depends strongly on how many times you've updated Ubuntu since you installed it. Each time a file in the OS is replaced, the new file is written in free space before the old one is deleted. Then other files can be written into the space freed by deleting the old OS file. That means your files will mix with the OS more and more the longer you keep the install. – Zeiss Ikon Sep 8 '17 at 12:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.