2

The problem: Thunar claims that it's written my files to a thumb drive, then I spend 15 minutes waiting for the drive to unmount.

I understand it has to do with the write cache. I hope there's a way to get it to do what I'd like, which is for the file manager to show me an accurate progress bar of the remaining write time (with an option to cancel the copy!)

What I'm unclear about is:

  • Does Ubuntu detect some drives as removable and apply optimal write cache settings?
  • If yes, how can I force it for my drive?

And, in general, ie, teach a man to fish:

  • How can I tune the write cache for a drive? The ideal write cache would be a small write-back cache that is written to disk as quickly as possible. Actually, the cache itself need not be small, it just has to permit a small number of dirty pages.
  • How can I have this setting apply each time I use a USB drive?

Full disclosure: I'm on Mint 17.2 Xfce

  • Is it the Gthumbnailer thats creating thumbnailes for the files on your USB stick? – Ken Mollerup Oct 29 '15 at 12:09
  • @KenMollerup I'm not sure. Would it matter? I have enough RAM that the files will stay in the disk cache so that making thumbnails from them shouldn't cause any I/O. – Aleksandr Dubinsky Oct 29 '15 at 12:39
  • Well on my system whenever i have moved or updated an external disk, and try to unmount or eject, it claims its busy, it turned out it was just creating thumbnails of pictures documents pdfs and such, and that can take a very long time, flushing the cash should be an ongoing process - and not delay you for a quarter of an hour. – Ken Mollerup Oct 29 '15 at 12:47
  • Re. "Thunar claims that it's written my files to a thumb drive, then I spend 15 minutes waiting for the drive to unmount." Could you please explain how you monitor the time taken to unmount? – Justice for Monica Oct 29 '15 at 13:49
  • @DKBose I click the unmount icon in Thunar. A message pops up similar to "we're writing your data, don't disconnect your drive." I see in my Disk Performance Monitor panel widget that the disk is being written. – Aleksandr Dubinsky Oct 29 '15 at 14:01
3

The settings that control how many dirty pages are allowed to accumulate before being written to disk are controlled by the virtual memory subsystem.

For systems with a lot of RAM, they usually need to be tweaked. Unfortunately, these settings are system-wide. (Ugh, Linus.)

See the current settings with:

sudo sysctl -a | grep vm.dirty

On my machine:

vm.dirty_background_bytes = 0
vm.dirty_background_ratio = 5
vm.dirty_bytes = 0
vm.dirty_expire_centisecs = 3000
vm.dirty_ratio = 10
vm.dirty_writeback_centisecs = 500

Take a look at the documentation for these settings. On my machine, vm.dirty_ratio is set to 10 (ie, 10% of RAM). This means that the disk cache will swallow up to 10% x 64GB = 6.4GB of data before writing it to disk.

I am going to try tweaking these settings by writing the following into my /etc/sysctl.conf:

vm.dirty_bytes = 67108864 
vm.dirty_background_bytes = 1048576
vm.dirty_expire_centisecs = 30
vm.dirty_writeback_centisecs = 30

I'll update this answer as I learn more, but I think these aggressive settings are fine for my SSD, which has its own large on-drive write cache.

1

I'm not sure if I can answer you what you want to hear. There is the "oldschool" option to disable that buffer - you may use the fstab sync attribute for this.

This will bring you away from the modern way of your device lying to you "your quick computer has already finished" when it hasn't. The reality still is "it's done when it's done" and sync will let your computer show you when it is really done.

If you in fact are annoyed by the "progress bar" (or waiting times in the shell) and want to tweak the buffer size and behaviour you may do that by editing the sysfs/block options. Compare https://unix.stackexchange.com/questions/30286/can-i-configure-my-linux-system-for-more-aggressive-file-system-caching for this. The list of availible options may vary depending on the ioscheduler you(r kernel) use(s).

Now the question is: How to apply this for USB sticks.

AFIAK on Linux Mint udisks handles USB Stick automounting and AFAIK there is no elegant way to teach udisks sensible defaults. You may try udisks-glue to help you.

If there is only a limited number of USB sticks you want to tweak in detail, you may write an udev rule to do set the mount or sysfs options you want. I always look at https://wiki.archlinux.org/index.php/Udev#Writing_udev_rules whenever I'm tinkering with udev.

Hope that helps

  • Thank you for the detailed answer! Would it change in any way for vanilla Ubuntu? Does it also use udisks? – Aleksandr Dubinsky Oct 29 '15 at 11:59
  • I'm trying to follow the advice in your link, but sudo echo 1 > /proc/sys/vm/dirty_background_ratio gives Permission denied. Moreover, this seems like the key setting that I'm looking for, but is there any way to specify it per drive? – Aleksandr Dubinsky Oct 29 '15 at 12:04
  • You may write that variable using sysctl, i.e. sysctl -w vm.dirty_background_ratio=1. Although I don't suggest that - this is a central part of the system and the consequences are not only per drive. In case your system freezes you should be fine after reboot. Just don't have unsaved important work. – Nachtexpress Oct 29 '15 at 14:20
0

User Nachtexpress' advice regarding sync is correct. As to how to implement this dynamically, use/configure Autofs as per the instructions at the Debian Wiki for this service (link: https://wiki.debian.org/AutoFs).

You will note that you can set sync option for USB drives in a config file as per the following syntax and line:

 /var/autofs/removable   /etc/auto.removable --timeout=2,sync,nodev,nosuid

The above is just an example you can configure to your specific needs; go to wiki link for full instructions on how to do this.

I think the debian wiki is appropriate for ubumtu users as this distro derives from it.

Hope this helps.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.