Defragging NTFS Partitions from Linux

Question

Are there any tools that I can use to defrag my Windows partition from Linux?

Answer 1

Yes, you can use shake. You'll first need to add a custom repository to your system:

sudo add-apt-repository ppa:un-brice/ppa
sudo apt-get update
sudo apt-get install shake-fs

Then you can do

sudo shake /some/dir

Answer 2

There is no such a tool around, for what I know.

Some site reports the following command

# WARNING - does not work
fsck -t ntfs --kerneldefrag /dev/hdX

but this does not work, and it is not clear where do they get it.

Answer 3

Update: UltraDefrag for Linux:

UltraDefrag is a powerful Open Source Defragmentation tool for the Windows Platform. It can defragment any system files including registry hives and paging file. Also one of the main goals of UltraDefrag is doing the job as fast and reliable as possible. It is being ported to Linux and NTFS-3G for defragmenting NTFS partitions. Currently only a test version in console mode is available. Please read the included file README.linux for compiling and testing

http://jp-andre.pagesperso-orange.fr/advanced-ntfs-3g.html

[I've not yet used this myself. Found it via a thread on an Arch forum. Further following the thread through to the next page leads to more on the topic. Try at your own risk.—kevjonesin—]

Answer 4

There is a very well know tricky cheat to archive a NTFS defragmentation (including free space)... it can be done with a Live Linux like SystemRescueCD...

But, you must have a second HDD or at least a 51% free space (that if not using compression).

The trick (very tricky) is to use tools to "clone" the partition, but not typical ones... there is a tool that can "clone" a NTFS parttion but not doing an exact "clone"... let me explain...

The tool is fsarchive (if i do not remember bad).

It makes a very big file (as clone tools does), with all files (beware of NTFS special metadata of files, streams i think they are callled), just as a "clone" tool does...

But the tricky part comes when restoring... it does not put files where they where, it puts files without any fragmentation at all.

I had used it for Windows System partition as well as for NTFS data partitions... with succed... not very easy to use / understand (read carfully the docs)... but can do the trick.

Remember... such big file it create can be compressed, so not really need to have a 51% free.

But any way, first you must Shrink NTFS partition... use GParted (if i dont remember bad).

So steps:

1. Worst case: HDD has only one BIG NTFS partition, with enough free space
2. Boot SystemRescuCD (if you want start X windows with wizard command)
3. Use GParted to shrink NTFS partition to its min size (will fragment NTFS files more, no matter)
4. Use GParted to create another partition on free space, better if ext4 type (journal)
5. Mount such new partition, but do not mount NTFS
6. Use fsarchiver to create a "clone" of the NTFS partition, use compression and store the bif file on created partition
7. From now on this is very RISKY: Use GParted to delete NTFS partition and re-create it
8. Restore the "clone" using fsarchiver, all files on NTFS partition will be 1 fragment (except one or two because $MFT could be putted on middle, not at strat or end of partition)
9. Use Gparted to shrink NTFS partition if needed
10. Check NFTS is correct, can be mounted, read/write, and dismounted
11. Unmount newly created ext4 partition
12. Use Gparted to grow that ext4 (or better, delete it and recreate it)

That's it, folks... as i said very tricky and risky, of course.

Beware Key Points:

• NFTS shrink let enough free space to hold a compressed image of the NTFS partition (51% free on the HDD will ensure there is enough)
• fsarchiver does not save NTFS streams?
• fsarchiver restore do not put files where they where, it put them unfragmented (like a copy)
• After deleting and recreating NTFS partition there is no back way

The steps are for: Use a LiveCD / LiveUSB linux to defrag a NTFS partition that uses 100% of the disk, with no other HDD connect to such PC [must have enough free space].

IMPORTANT:

• Some could think if you have 51% free on the HDD after NTFS shrink it could be safer to just create that ext4 and copy all NTFS to it, then delete NTFS, recreate it and copy them back... DOING THAT will cause looses, like NTFS security permisions, etc

As long as i know GParted/fsarchiver is the most near solution (for using only a LiveCD/LiveUSB with Linux), not fast, somehow risky, but satisfing terms imposed by the person who post the question.

BE AWARE: It is very, very risky... as with any "clone" tool... and for now i do not know any other best way... and it will loose NTFS streams (if i do not remember bad)

PLEASE: Read carfully fsarchiver docs, before using it.

I have any responsability (i had used it on my own data, but allways i have a external BackUP - good partices).

In practice... all info i have, i put them on at least three different mediums (HDD, DVD, Flash Memories), with at least three copies on each medium, so i have 9 copies... if i loose 8 of them it rest one more to recover data... i am a little paranoid, i kown.

Answer 5

This is a BIG warning for all those of you that think NFTS can be defragmented on Linux just by copying files (cloning only files), etc:

• Some NTFS (files/folders) can have a special attribute active, it is called NTFS Compression

from what I know, any time Linux (cp, fsarchiver, etc) writes a file/folder on a NTFS it always write it without NTFS compression, no matter if the file/folder has compression on or off.

So you can get to a situation (I meet it the hard way), where restoring with fsarchive (or cp, etc) would make partition to get full and not be enough.

Some kind of data can reach an NTFS compression ratio of more than 3, so you can have a X GiB partition with a lot of files, and the sum of files be near 3*X in size.

I give that warning because it is not well known and sometimes creates really big headaches. like when restoring a clone need more space than the whole partition that has been cloned, caused because NTFS compression got lost on Linux.

Also, with very very special data (NTFS ratio greater than 5) I reach this situation:

• NTFS partition size of X GiB
• The file that holds the clone (with the best compression that tool let, GZip I think) took 2*X GiB

Oh yes, the clone was compressed and it took double that partition size.

That is caused because the clone tool read files in plain (in clear, not compressed) then compress the data (with a really worst ratio than NTFS did).

Of course restoring that data will not fit on that partition, since restored data will be putted without NTFS compression.

Hope it is clear another reason why not to use NFTS compression? Well, not at all, I use NTFS compression a lot (in the past). VDI (Virtual Box) files get a really good ratio.

Now I had discovered Pismo file Mount (and it also works on Linux). it can create a file that acts as a container (as a folder) and can be compressed (also with better ratio than NFTS) and at the same time encrypted.

Why I mention it. because any clone tool will see such container as a file (when not mounted as folder) and read/dump/backup the compressed stream of data, not the plain uncompressed data (as with NTFS compression). so restoring is as with any other file.

Instead of compressing a NTFS folder with NTFS compression attribute, I put a Pismo file Mount virtual folder. get better compression, etc.

I must also warn all of you interested on such free tool. it has no shrink (at least yet), so if folder content changes a lot it is not a so good idea.

But for immutable Virtual Disks, ISOs, and things that will not change, the ratio it gets is very close to LZMA2 ones (7-Zip) and it can be read/write on the fly.

And it is cross-platform.

Note the bad guy of NTFS compression talking about fragmentation. when you write a file to a NTFS with NTFS compression on, it does it this way (yes horrible designed, I think it is done like that to ensure greater fragmentation in intention way, worst can not be done):

1. Start write position is pre-calculated as 64K*N, where N is the number of the 64K chunk that will be tried to be compressed
2. A buffer for 64K is reserved
3. That buffer is filled with 64K and then compressed
4. Only the 4K blocks needed are written, the rest are let as free space

So it creates a lot, lot of GAPs in the middle of the file, and only after a file defragmentation that GAPs disappear, but that defragmentation does not occur until user order it (contig.exe, defrag.exe, etc).

Yes, it writes the N'th 64K chunk on a position multiple of 64K, no matter if previous data could or not could be compressed, it leave a Gap between each 64K chunk (if all could be compressed).

Pismo File Mount virtual folder compression acts like any normal compression is supposed to be done, piped mode, so no gaps. at least until you delete something.

Also another warning, do not put VHD / VHDX files inside it, Windows will not be able to attach them! Windows uses a kernel trick to mount such things, it does not use file-system level, works at low level.

I would also like to get my hands on a Linux NTFS defragmenter, sure would be faster than all that run over Windows. it is a total madness to defragment free space. or better talking. creating a whole big enough for a new big file.

Also it would be great my memory work better. in the past I was using a tool (command line, sorry) on Windows that could copy/move a file in non-fragmented way. moving the needed files away while getting that needed whole, and not fragmenting that ones. it only gives a message if it can not find a way to put the file (impossible to get a hole) or a different warning if it needs t fragment another file (asking if authorised by user), etc. was really great. I did not remember the name (and maybe it does not work with modern windows, it was for Win2000).

Answer 6

No. These are Windows filesystems so you will need to use Windows to defrag them.