1

I am using Xubuntu 16.04 on my laptop, installed from a dist-upgrade of a Xubuntu 14.

The laptop has 10 GB RAM and 4 GB Swap.

Recently I upgraded the kernel to version: 4.15.0-51.

The problem is that during normal operation the system is swapping-out over time but never swapping back in. So after hours/days of uptime, the swap space gets full.

Even after closing all the main processes the RAM usage goes to a few hundred MB but the swap remains full. Also, looping over all the /proc/*/stats and grep VmSwap return 0 kB for all processes.

How can I debug why the kernel is not swapping in?

It's like the kernel is losing track of what processes the pages on the swap belong to.

I've already invested a significant amount of time trying to fix this with no success. The only turn around I have for now is a script that is creating and recycling (swapoff ; swapon) a few swap files.

Thank you for any help.

2
  • 1
    Possible duplicate of How to empty swap if there is free RAM? – Fabby Jun 19 '19 at 12:50
  • Not exactly the same. I can live with this issue, have a script to run "sudo swapoff -a; sudo swapon -a", in fact, I made a script to manage a few swap files to be sure I don't get caught in the irresponsive world of an "out of RAM and out of Swap". What I want to point out is that I have 3 machines using Xubuntu 16.04 and on two of them if I close firefox, for example, I see the swap being swapped-in, however on the third machine that does not happen, even if it has 1BG used RAM and 9GB free RAM. I really think that this machine has a software issue but I am falling to debug it. – Fininho Jun 19 '19 at 17:26
1

Today I gave it another try and found the problem.

Started by checking again the meminfo information. Here is the output of cat /proc/meminfo:

MemTotal:       10133276 kB
MemFree:         2617372 kB
MemAvailable:    4660420 kB
Buffers:          503472 kB
Cached:          3825884 kB
SwapCached:        59448 kB
Active:          3852624 kB
Inactive:        3112996 kB
Active(anon):    2688920 kB
Inactive(anon):  2245220 kB
Active(file):    1163704 kB
Inactive(file):   867776 kB
Unevictable:          32 kB
Mlocked:              32 kB
SwapTotal:       6096888 kB
SwapFree:        2644324 kB
Dirty:                52 kB
Writeback:             0 kB
AnonPages:       2576972 kB
Mapped:           770460 kB
Shmem:           2297836 kB
Slab:             391072 kB
SReclaimable:     325960 kB
SUnreclaim:        65112 kB
KernelStack:       15824 kB
PageTables:        53492 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    11163524 kB
Committed_AS:   12259512 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:      2048 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      349640 kB
DirectMap2M:    10039296 kB

Despite the issue of having 4 GB of swap used by nothing, this time the 2GB of shmem used caught my attention.

Searching for that, I found this question with the answer: High SHMem memory usage!

So, in short, having high shmem usage can be related to tmpfs directories. In my case, /dev was using almost 4.8 GB

Here is the output of df -h:

Filesystem      Size  Used Avail Use% Mounted on
udev            4,8G  4,8G     0 100% /dev
tmpfs           990M   34M  956M   4% /run
/dev/sda2       255G  232G   11G  96% /
tmpfs           4,9G  220M  4,7G   5% /dev/shm
tmpfs           5,0M  4,0K  5,0M   1% /run/lock
tmpfs           4,9G     0  4,9G   0% /sys/fs/cgroup
/dev/sda5       333G  208G  126G  63% /media/dados
cgmfs           100K     0  100K   0% /run/cgmanager/fs
tmpfs           990M   88K  990M   1% /run/user/1000

After that I just searched in /dev for the culprit using the du command. As also referred to in the thread I linked above, it was the bootchart command that was storing huge log files inside the /dev directory.

Thanks for the help.

0

You don't need to worry about it. Have a look for example here. Kernel is tracking it and it can reause space in swap.

2
  • You have enough rep to flag the question as a duplicate.... – Fabby Jun 19 '19 at 12:51
  • That's the point, I have two other machines were that is true, but on this one, I had been caught by out of RAM and Swap too many times. I think that on this machine the kernel is falling to tracking the swap pages, but I don't know how to debug it. Do you know how can I increase the verbose on the service that manages the memory/swap? – Fininho Jun 19 '19 at 17:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.