I have some swap configured and I believe activated...
Filename Type Size Used Priority
/dev/sda5 partition 7811068 1124912 200
/mnt/data02/swapfile file 134217724 37032 100
/home/swapfile file 134217724 36600 -1
But, when the system monitor shows memory reaching 100% the system tends to respond by closing/crashing programs. Xorg has crashed in this way as has the wireless driver. While this happens the system monitor shows little (under 5 GiB) use of the Swap. I have confirmed that swappiness is not set to an extreme value (and changes in this parameter have appeared to have no effect on the issue).
~$ cat /proc/sys/vm/swappiness
70
The system has a huge amount of RAM...
~$ free -h
total used free shared buffers cached
Mem: 125G 20G 105G 161M 54M 1.1G
-/+ buffers/cache: 19G 106G
Swap: 263G 1.1G 262G
... but sometimes I run overbudget on Memory resources and it would be nice if it failed more gracefully than things crashing.
What can I do to resolve this issue?
EDIT
~$ cat /etc/fstab
# <file system> <mount point> <type> <options> <dump> <pass>
# / was on /dev/sda3 during installation
UUID=8dfbed62-9957-4f06-b4e1-a42020adec91 / ext4 errors=remount-ro 0 1
# /home was on /dev/sda6 during installation
UUID=b6f33408-1d8b-4302-9983-5c778ef64f47 /home ext4 defaults 0 2
# swap was on /dev/sda5 during installation
# ae0304dd-e63e-4d3a-99da-9c9d7a034c6e is the swap file
UUID=fd4c00c9-49bf-4562-adea-1c817fc57ce9 none swap sw,pri=200 0 0
UUID=3A323DCA323D8BBF /mnt/data01 ntfs-3g defaults,windows_names,locale=en_US.utf8 0 0
UUID=4cc8a19d-5991-4186-8f65-7062805b66a6 /mnt/data02 ext4 defaults 0 0
/mnt/data02/swapfile none swap sw,pri=100 0 0
/home/swapfile none swap sw 0,pri=150 0
EDIT 2
In response to a comment below I watched my system perform an operation I knew would use all available RAM but not all available swap and then checked dmesg
during and after the failure. The system swapped and became intermittently non-responsive (okay behavior). Then while swap was at less than 10% of capacity, Chrome crashed (Sorry, the program "chrome" closed unexpectedly. Your computer does not have enough free memory to automatically analyze the problem and send a report to the developers
). Trying to get back to the dmesg output I was watching I got an error message that stated This window is not responding. Do you want to force the application to exit, or wait for it to respond
. I selected 'wait'. The desktop reappeared and the gnome system monitor went in and out of being greyed out several times while the system swapped. When I checked back in, I was at the Ubuntu login screen. I logged in as normal... all of my earlier running processes were gone and I was greeted with an error message identical to the one about Chrome in reference to Xorg. Checking dmesg shows only the following two messages:
[131267.206774] Watchdog[3433]: segfault at 0 ip 00007fe38faf9756 sp 00007fe37f393770 error 6 in chrome[7fe38be0a000+510c000]
[133329.875212] nvidia 0000:03:00.0: irq 106 for MSI/MSI-X
EDIT 3 Other possible relevant topics:
- The OOM killer may be called even when there is still plenty of memory available, although I still am not sure how to check if OOM was called.
- Perhaps some configuration of pages_low / min_free_kbytes would get me to where I need to be, more directly related than swappiness, but I'm not seeing what [some sites suggest I should find in /proc/zoneinfo (I can't post a link: mariosmarduch.ulitzer.com/node/431838/mobile), but that may be because they aren't Ubuntu specific?
- I can't post a link: www.linuxdevcenter.com/pub/a/linux/2006/11/30/linux-out-of-memory.html?page=1
Edit 4 Additional error messages:
[92315.165728] Watchdog[1319]: segfault at 0 ip 00007f7d0a417756 sp 00007f7cf9cb1770 error 6 in chrome[7f7d06728000+510c000]
[92656.478271] INFO: task Chrome_IOThread:1292 blocked for more than 120 seconds.
[92656.478275] Tainted: P OX 3.13.0-45-generic #74-Ubuntu
[92656.478276] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[92656.478278] Chrome_IOThread D ffff88207fc534c0 0 1292 32756 0x00000000
[92656.478282] ffff881fa9d15dd8 0000000000000086 ffff881fad2ab000 ffff881fa9d15fd8
[92656.478285] 00000000000134c0 00000000000134c0 ffff881fad2ab000 ffff881fad2ab000
[92656.478288] ffff881f5fce6260 ffff881f5fce6268 ffffffff00000000 ffff881f5fce6270
[92656.478290] Call Trace:
[92656.478299] [<ffffffff817252d9>] schedule+0x29/0x70
[92656.478302] [<ffffffff81727f55>] rwsem_down_write_failed+0x115/0x230
[92656.478307] [<ffffffff81371d63>] call_rwsem_down_write_failed+0x13/0x20
[92656.478311] [<ffffffff81314c90>] ? apparmor_file_mprotect+0x30/0x30
[92656.478313] [<ffffffff8172796d>] ? down_write+0x2d/0x30
[92656.478318] [<ffffffff8116ba7c>] vm_mmap_pgoff+0x6c/0xc0
[92656.478322] [<ffffffff8117f916>] SyS_mmap_pgoff+0x116/0x270
[92656.478325] [<ffffffff81018802>] SyS_mmap+0x22/0x30
[92656.478328] [<ffffffff8173196d>] system_call_fastpath+0x1a/0x1f
mmap(...,MAP_LOCKED)
) just in RAM, non-swappable, etc. Both chrome and the graphic drivers do have suid parts (look at chrome sandbox). So you can have an OOM condition even with swap available --- just for this kind of memory. I do not know if this can trigger the OOM killer, or just the allocation fails and the process kill itself -- this is why this is just a comment and not an answer.