How can I do a disk low level formatting in Ubuntu on an external hard disk so that any data recovery tools like test-disk or photorec cannot retrieve any data once formatted to ext4 or any file-system? As I was playing around with Test disk, I found that it can recover almost all old files even after formatting it many times and I want to use my external hard disk as new because it is only one year old and in warranty but without any old data.
Modern discs implement the ATA secure erase
feature, which you can do with the
hdparm command using the
--security-erase option, after first setting a password on the disk. Note that there are caveats, including
- possible firmware bugs
- possible disk controller timeout for a possibly long-running operation
- you shouldn't do it over USB
For some discs this will take hours, as each block is rewritten. For others it can take seconds as it just means changing a global encryption key held by the disc that transparently encrypts/decrypts all data going to/from the disk. This is true for hard discs and SSDs. It's the firmware than counts.
If you want totally wipe the hdd the best is to use
dd if=/dev/zero of=/dev/sdX bs=512
Replace sdX with your drive letter
This command is a replacement for rm command. It works under Linux/BSD/UNIX-like OSes. It removes each specified file by overwriting, renaming, and truncating it before unlinking. This prevents other people from undelete or recovering any information about the file from the command line. Because it does lots of operation on file/directory for secure deletion, it also takes lot of time to remove it.
This doesn't really answer your question, as you want to use the HDD. However, it's important.
If this is really important data that should never be recovered, it's not safe to use the disk any more. Apple Macs offer a 35 overwrite feature - which they claim is what the government requires, but it's complicated:
Effective immediately, DSS will no longer approve overwriting procedures for the sanitization or downgrading (e.g. release to lower level classified information controls) of IS storage devices (e.g., hard drives) used for classified processing.
It is thought that some of the 3-letter agencies (FBI, CIA, MI6) require physical destruction of magnetic media (e.g., melting in a furnace).
You do have a few options - the one I've relied on in the past is hitting it with a hammer lots of times, then wiping the magnets over it.
However, I'm a student and nobody wants to hack into an empty bank account, so it didn't need to be as secure. Plus, I wasn't going to bin the Hard Drive - it's now a bird scarer for the allotment.
If you work under confidentially agreements (especially those involving the government) you need something a little more secure. I'd recommend a blowtorch:
My pronouns are He / Him
dd command by Maythux is a good one to go with, though I've read somewhere (sorry for no source!) that it's good to overwrite with zeros then do a second pass with random writes, then zero it off on a third pass.
dd to write random patterns:
dd if=/dev/urandom of=/dev/sdX bs=512
I have seen some examples of dd using smaller block size (down to bs=4) which I think will make the write slightly more random, but take longer.
Be super careful that you select the correct drive when using dd as it's very dangerous if you make a mistake!
NaA, but some additional informations, expecially about SSDs
There are lots of methods and lots of debates around the topic; basically this is about the type of the support (magnetical disk or flash disk) and about the sensitivity of your informations.
Data which are really sensitive might be considered undeniably safe only when the physical drive is destroyed, since, theoretically [but not too much] (placeholder for the link if I ever manage to find it again) laboratory methods capable of measuring differences in the magnetic / electronic fields of a support, joined with the knowledge of the algorithm used by firmware of the drive to handle the data might be able to extract even data that has been overwritten multiple times.
However, for the purpose of erasing a hard disk containing personal data, each of the methods above will work fine against a software recovery attempt; it might be noted, however, that methods using low-level utilities are not good to be used on SSDs, for multiple reasons.
The first one being that writing a flash-based memory (expecially multiple times) is not really healthy for the memory itself, altough the wear-leveling issue is highly overestimated (expecially on MLC SSDs), i.e.: you indeed don't want to
dd a SSD drive every day, but once, twice or even thrice (or even more) in its whole life-span is absolutely not a big deal;
The second one being the fact that commercial SSDs usually come with additional replacement cells, which are initially not being used by the drive, and which are intended to act as a replacement for cells being damaged during the usage of the drive; this leading to, possibly, the SSD controller unmapping such damaged cells (possibly containing sensitive informations) at some point of the drive's life making them physically inaccessible from low-level tools.
So, if you're using a SSD, the best solution is probably sticking to the manufacturer's ATA
SECURE_ERASE command implementation (as described in meuh's answer), hoping that this will be robust enough for the purpose.
I want to elaborate on a detail in the question that no other answer has touched upon: low-level formatting. In the past, a distinction was made between two types of disk formatting:
- Low-level formatting -- This refers to creating the data structures that define the sectors and tracks on a disk.
- High-level formatting -- This refers to creating a filesystem on a disk.
Some decrepit old fools like me still twist our sentences into pretzel shapes to avoid using the word "formatting" alone because that's ambiguous. For years, in the Unix/Linux world, high-level formatting was always referred to as creating a filesystem. The word "formatting" has been creeping into various utilities in this context, though.
It's important to note that, to the best of my knowledge, no modern hard disk supports low-level formatting by users; that task is done at the factory and cannot be done again once the disk leaves the factory. Hard disks in the 1980s and maybe into the early 1990s did support low-level formatting by users, but those disks are all museum pieces by now. You can still low-level format a floppy disk, if your computer is old enough to actually have one. This was often necessary with floppies, since different computers (Macs vs. PCs, say) used different types of low-level formats, so if you got a Mac floppy and wanted to use it on a PC (or vice-versa), you'd have to low-level format it.
Also and FWIW, I looked into it a few years ago, and I could find no references to actual tools that could be used to recover data from a disk that had been wiped by using
dd to zero out all its sectors. It's theoretically possible to do so by disassembling the disk and using special hardware to read it. If the disk holds national security secrets, going beyond
dd might be advisable (and is required by many national security agencies); but I wouldn't worry about protecting your credit card numbers and Flickr password with anything beyond
dd, at least on a hard disk. SSDs use fundamentally different technology and I've not looked into them in any depth.
If you want absolute peace of mind (and still want to use it afterwards) give it the Guttman method.
It is overkill. A standard three-pass overwrite - clearing with zeros, filling with ones, then overwriting with a random string of either - is industry recognised and still the defacto means to certifiably sanitize a hard drive. If your drive is without bad sectors and the eraser can access all addressable areas - your data is gone.
It might be worth noting that there are no trading names which claim to be able to recover previously overwritten data.
Infact, there are good reasons why a single pass might well be enough. There is really only one clinical circumstance where data could "theoretically" be recovered under laboratory conditions: by using 'some sort of' molecular magnetic microscopy on what would have ideally been a previously clean (and preferably intact) drive - measuring the minute differences in residual charge thought to represent previous logical values and then reconstructing bit by bit with the aid of probability analysis. It is vague and deemed to be so prohibitively expensive - especially for newer hard drives - that it is questionable whether some agencies could be capable of even justifying the attempt. Your data certainly isn't worth it!
Obsolete hard drives are thought prone - but beyond one pass the chances of recovery approach zero.
The belief that this can be reliably done by any agency is very likely a myth allowed to perpetuate to overstate a nation's perceived range of capabilities.
Conversely, the mere suggestion that such data recovery methods are no longer (or never were) possible on (post-modern) hard drives is quite possibly the rumour which enables agencies and nation-states to keep the capability unannounced.
So if you want to, give it Guttman's ridiculous 35-pass wipe. Even more, encrypt it first and throw away the key (aka crypto-erasure - a method sufficient on its own). If you want total peace of mind - shatter the platters. If you're sane - wipe it three times, or just once if you're going to be actively using it again pretty soon.
I think it is maybe not as secure as professional tools or
dd or something like that, but if you just need to overwrite the whole file system's data part (! not disk), you could just format it normally and then create a huge file filling up the whole space with
yes "string pattern" > /mnt/device-to-kill/huge-file
This will create/replace the file
/mnt/device-to-kill/huge-file, assuming that your external device is mounted as
The content will be
string pattern overly and overly repeated. If you don't give a pattern as argument, default is
y. The command will not stop writing data until either the disk is full or you abort it with Ctrl+C.
The advantage of this method is that even an unexperienced user can't destroy anything accidentally. If you specify a target file on your HDD, it will create/replace that one and fill up every free byte of disk space you have, which should at most produce a crash and minor loss of unsaved data. It does not touch existing files.
This is also a disadvantage though, as you have to format the file system before to delete all files normally, as the command only overwrites free space. Besides this, it can only overwrite the data part of your partition/device, not parts reserved for e.g. file tables (FAT, MFT, whatever depending on your file system).
Conclusion: This command is more for newbies that fear to accidentally destroy more than they want instead of experts that need 110% secure data destruction.