4
  • Ubuntu 18.04 LTS
  • exfat-fuse / exfat-utils 1.2.8-1 (The latest version that will install after running sudo apt-get update.)

Linux is writing files to a Windows exFat partition on a Windows OS disk which are INVISIBLE to Windows. How do I make these files visible to Windows? (Update - further below, Linux subsequently deleted the files, 400GB of them, and as as you will see, Windows apparently didn't do it.)

I used Photorec, from Linux, to recover files to this exFat partition.

The exFat partition is 62% full, but Windows reports it as being empty, with all the space free (!!!).

I rebooted into Linux (external SSD w/Linux,), and mounted the exFat partition again to make sure the files were actually there. (That's a 700GB partition, and there is nowhere else that could come close to holding that data). Linux sees the files on that exFat partition.

I ran chmod 777 FOLDER/ -R, and then checked in Windows again. Windows still reports that the partition is empty (!!!).

The TestDisk documentation on Photorec says that Windows may need to take ownership of the files to access them, but the link they give says how to change the permissions of the recup_dir.1 (.2 ...) directories from WITHIN Windows - where they are not viewable (and 100% of the space is available !!!). That document is here: https://www.cgsecurity.org/wiki/PhotoRec_FAQ

How do I make these files viewable in Windows?

Thanks for any help.

UPDATE

I tried the suggestion in the TestDisk documentation, even though I can't see any folders from Windows. (The doc assumes recup_dir.1 will be viewable from Windows.) I tried to take ownership of the partion in Windows, rather than a directory.

[Administrator command prompt]
C:\WINDOWS\system32>takeown /f e: /r /d y
ERROR: File ownership cannot be applied on insecure file systems;  
there is no support for ACLs.  

C:\WINDOWS\system32>

So this isn't the problem, as it isn't relevant to exFat.

I dare not write anything to this partition from Windows, or do anyting else with it from Windows, as Windows believes it is empty. Photorec ran for 5 hours creating those restored files.

UPDATE

I ran another test. I copied a text file, created by a regular user account (not root or PhotoRec), to the exfat partition. Windows still doesn't see any files there.

UPDATE

Here are some images which show the problem. It also shows what happened after I ran chkdsk, without repairs being performed.

sda is the internal (Windows) disk.
sdb is an external SSD w/Linux.

lsblk output, showing mounted partition

directory listing in exfat partition

GParted, showing 400GB is used

Now, lets reboot and see what Windows shows.

Windows show 0% of the partition is used.

WINDOWS REPORTS THAT THE PARTITION IS EMPTY !!!

Now, run chkdsk, without selecting to repair the partition.

Output of chkdsk, no files, no errors.

Now, select repair. (Because I'm about to give up anyway, and rerun PhotoRec with this partition formatted as NTSF)

Windows says the partition is locked, I deny forcing a remount

Windows says the partition is locked. Now I'm going back to Linux.

Output from some Linux commands.

THE FILES ARE GONE. 400GB of them.

I had gone back and forth between Windows and Linux a few times, trying to sort out this problem, and Linux was still able to see the files.

exfatfsck reports there are no errors, and the partition is empty - except for perhaps 1 file. I ran exfatfsck on a new exfat partition I created as a test, and it reports the empty partition has 3 directories and 2 files. So there might be one file there, but it isn't showing in the directory listing.

I ran testdisk on the partition. It reports
Bad GPT partition, invalid signature. Trying alternate GPT Bad GPT partition, invalid signature.

I ran scan (testdisk), and let is run a bit (it would take hours). It found some files that are non-exfat (I guess left-overs, from before I cut out that partition for this. I did a quick format.). I could have let the full scan run, but I didn't bother. I can run PhotoRec again.

The Work-around

I researched this for a couple of hours. I found two posts that describe very, very similar problems, and lots of posts saying that exfat is dangerous, and linux has much better support for NTFS. Also, it is very easy for an exfat partition to get damaged, as opposed to NTFS.

So, I am remaking this partition as NTFS, and running PhotoRec again.

I believe this is very important lesson about NOT using exfat. Fortunately, in my case it just wasted a lot of my time.

The Question
Does anybody know how to get a shared exFAT partion working, or what happened here? Maybe it's an Ubuntu 18.04 LTS issue. It would be nice to be able to use exFAT for this, because with NTFS I have to turn off fast-boot, and avoid hibernating Windows. (Maybe there's a work-around for that... That's a different question.)

Improve this question
8
  • Comments are not for extended discussion; this conversation has been moved to chat.
    – Thomas Ward ♦
    Jan 3, 2020 at 18:05
  • I added significant information and images describing this Ubuntu 18.04 LTS and/or exfat-utils/exfat-fuse problem. The moral of the story is, don't use exfat for sharing a partition between WIndows and Linux. I hope this post gets restored, as it's important that people see this. I'll be redoing this work using NTFS.
    – JasonF4
    Jan 3, 2020 at 21:20
  • OK, you’ve persuaded me.  Good job on gathering and presenting evidence.  A couple of points (that illuminate your problem, but, unfortunately, don’t lead to a solution): (1) You observe that exfatfsck says that there are 2 or 3 files, but you can’t see them.  Have you done an ls of System Volume Information?  There are probably files in there.  (2) You say “The files are gone. 400 GB of them.”  I presume you noticed that exfatfsck says that there are still 406 GB used, right?  (I don’t understand why you say that Linux deleted the files.) … (Cont’d)
    – G-Man Says 'Reinstate Monica'
    Jan 6, 2020 at 19:01
  • (Cont’d) …  (3) People are saying that this is entirely a Windows problem.  I’m not convinced.  It seems to me that the problem is that the exfat-fuse / exfat-utils 1.2.8-1 driver is writing to the exFAT file system incorrectly, so this is, arguably, a Linux problem.  But that is probably as far as it will go.  It might be interesting to hear whether anybody is getting this software to work.  If there are newer versions, you will be told to try them, if you can.  You will be advised to contact the developers, if you can find them — ask for help and/or file a bug report.  … (Cont’d)
    – G-Man Says 'Reinstate Monica'
    Jan 6, 2020 at 19:01
  • (Cont’d) …  (4) terdon said “Linux is very good at reading other filesystems. But you're trying to do it the other way around”. I believe that terdon meant (but failed to express clearly) that, while Linux is very good at reading foreign filesystems, it is much less good at writing to them. So I agree that this is the problem you are facing. (5) It makes sense for you to abandon this experiment and just use NTFS. If you do that, people here will not want to continue to help you with the exFAT problem. On the other hand, if you pursue that problem, I encourage you to leave PhotoRec out of it.
    – G-Man Says 'Reinstate Monica'
    Jan 6, 2020 at 19:01

4 Answers 4

Reset to default
3

Does anybody know how to get a shared exFAT partion working, or what happened here?

It could be a bug. Linux exfat is still new and ntfs-3g took years to get really stable.

It would be nice to be able to use exFAT for this, because with NTFS I have to turn off fast-boot, and avoid hibernating Windows.

That's probably your problem, then.

ntfs-3g does not prevent you from mounting a suspend/hibernated Windows partition out of spite, it's trying to save you from doing something very dangerous that'll inevitably lead to filesystem corruption.

If you want to share a single filesystem between two OSes, you have to disable suspend/hibernate/fastboot/watchamacallit on BOTH sides. Doesn't matter if it's Windows and Linux, or Linux and another Linux.

See also https://www.kernel.org/doc/Documentation/power/swsusp.txt

* BIG FAT WARNING *********************************************************
*
* If you touch anything on disk between suspend and resume...
*             ...kiss your data goodbye.
*
* If you do resume from initrd after your filesystems are mounted...
*             ...bye bye root partition.
*         [this is actually same case as above]

So this is not filesystem specific but a generic problem.

Suspend-to-disk (hibernate) never really shuts down the system. It dumps memory state to disk and that memory state includes mounted filesystems and the like. So filesystems are still mounted even when the machine is "powered off".

The only thing you are allowed to do in this state is "resume". If you do anything else, you will very likely lose data.

Suspend one OS, then boot into another OS instead of resuming, and change data on disk... then reboot the first OS and resume. Memory state is loaded and set back in time, the system keeps running as if it was never shut down in the first place, it is not aware of disk changes made by another OS in between.

It's the same principle as using dd to write into a mounted filesystem partition. A filesystem can only be used by one entity at a time, if you make it two (Linux kernel and dd, Linux kernel and another Linux kernel, Linux and Windows, vice versa), they'll just end up corrupting each other.

Improve this answer
2

For me the issue I think it may have been a filesystem hibernated / locked issue. I say that because I booted in and out of Win10 and Ubuntu 18.04 several times, checked msftdata flag was OK, same deal, then I added a text file to an "invisible to Win10" folder using Ubuntu, just now I reboot into Win10 and bam!, everything appeared.

All copying and mounting was done via Ubuntu 18's std file explorer... Nothing special done really, just plug and play. Cheers

Improve this answer
1
  • To clarify, when an OS is hibernated, it is still keeping in memory metadata about the filesystem - and it is not programmed for the possibility of the data changing before it resumes - what is what can happen if you boot into another OS - so if you modify files and then resume a hibernated OS, it might write to the drive as if it was not modified, and this can cause all sorts of corruption. I think the OP forced write access. The warnings are there for a reason, read them! :-D
    – Dagelf
    Nov 6, 2020 at 6:45
0

I had trouble with exFat partitions moving from one Ubuntu 20.04.1 LTS box to another. I made a backup with simple cp commands after deciding against archives which could get big.

    cp -p -u -r -v Data/ /media/username/Backup 
    cp -p -u -r -v datamodeler.log/ /media/username/Backup 
    cp -p -u -r -v Desktop/ /media/username/Backup

As this was a backup of all my data, projects, and documents I let it run overnight. In the morning I resumed my machine, unmounted the disk and tried it in the new machine.

The phenomena was the same as reported above. The directory and file structure appeared, but no file content appeared.

I repeated this a few times, removing and recreating exFat (type 07) partitions, poking at it with fdisk command line, Disk utils. I was beginning to wonder if the media or devices were bad, but now having done a limited test with just 20GB and no chance for the source machine to sleep, I have the files. Media files play, etc.

I chalk this up to the suspend / hibernate / resume issue. I think the format is OK.

Note no Windows OS ever touched my files.

Improve this answer
0

I'll post this in case someone finds it useful. I had almost the exact same setup/behaviour that OP reported; the only noticeable difference was that the exFAT partition was created with Windows instead of Linux.

In my case chkdsk /f DID make the files visible to Windows

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .