2

I have problems with huge RAM consumption by an unknown entity. It is not listed in top, but the amount of memory available decreases over time and turning off all programs does nothing. Currently, it's 3.6 GiB, but it's slowly getting worse. Probably as a result, the system is not capable to swap out much memory, after swapping out about 1.5 GiB of memory, it proceeds to thrashing.

I have used the free -m command to get this result:

              total        used        free      shared  buff/cache   available
Mem:           7875        3517         388        3595        3969         481
Swap:          9536         500        9036

It seems that shared memory takes 3.5 gigabytes.

This is what I get when I run df -h:

udev                 3,9G     0  3,9G   0% /dev
tmpfs                788M   80M  709M  11% /run
/dev/sdb6             23G   18G  4,1G  81% /
tmpfs                3,9G  168M  3,7G   5% /dev/shm
tmpfs                5,0M  4,0K  5,0M   1% /run/lock
tmpfs                3,9G     0  3,9G   0% /sys/fs/cgroup
/dev/sdb1            356M   76M  281M  22% /boot/efi
/dev/sdb8             37G   33G  2,0G  95% /home
/dev/sda2            746G  269G  477G  37% /data
/dev/sda3            184G  132G   43G  76% /lindata
tmpfs                788M   84K  788M   1% /run/user/1000
/home/dugi/.Private   37G   33G  2,0G  95% /home/dugi

/proc/meminfo/ contains this, it seems that it's really something in shared memory:

MemTotal:        8064776 kB
MemFree:          318880 kB
MemAvailable:     493992 kB
Buffers:           36784 kB
Cached:          3960388 kB
SwapCached:        32776 kB
Active:          3732236 kB
Inactive:        3539036 kB
Active(anon):    3562316 kB
Inactive(anon):  3381848 kB
Active(file):     169920 kB
Inactive(file):   157188 kB
Unevictable:         732 kB
Mlocked:             732 kB
SwapTotal:       9765884 kB
SwapFree:        9253044 kB
Dirty:               516 kB
Writeback:             0 kB
AnonPages:       3269552 kB
Mapped:           493840 kB
Shmem:           3670056 kB
Slab:             264136 kB
SReclaimable:     136840 kB
SUnreclaim:       127296 kB
KernelStack:       23296 kB
PageTables:        82464 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    13798272 kB
Committed_AS:   20782644 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:    831488 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:     1783952 kB
DirectMap2M:     6498304 kB
DirectMap1G:           0 kB

However, when I list the stuff in shared memory using ipcs --human, I get only a few hundreds of megabytes of deleted memory:

------ Message Queues --------
key        msqid      owner      perms      size         messages    

------ Shared Memory Segments --------
key        shmid      owner      perms      size       nattch     status      
0x00000000 268959744  dugi       600        386,8K     2          dest         
0x00000000 265879554  dugi       600          512K     2          dest         
0x00000000 557059     dugi       600           16M     2          dest         
0x00000000 6258692    dugi       600            1K     2          dest         
0x00000000 271974405  dugi       600          512K     2          dest         
0x00000000 31784966   dugi       600           16M     2          dest         
0x00000000 819207     dugi       600           16M     2          dest         
0x00000000 274464776  dugi       600          512K     2          dest         
0x00000000 28704777   dugi       600          512K     2          dest         
0x00000000 274694154  dugi       600          512K     2          dest         
0x00000000 264372236  dugi       600          512K     2          dest         
0x00000000 272465933  dugi       600            4M     2          dest         
0x00000000 145522702  dugi       600           16M     2          dest         
0x00000000 274563088  dugi       600          7,6M     2          dest         
0x00000000 113934353  dugi       600           74K     2          dest         
0x00000000 215908370  dugi       600         66,2K     2          dest         
0x00000000 35913747   dugi       600          512K     2          dest         
0x00000000 274595860  dugi       600          7,6M     2          dest         
0x00000000 149946389  dugi       600          512K     2          dest         
0x00000000 273842198  dugi       600          128M     2          dest         
0x00000000 115015703  dugi       600         29,3K     2          dest         
0x00000000 150044696  dugi       600           64M     2          dest         
0x00000000 143425561  dugi       600         32,5K     2          dest         
0x00000000 150011930  dugi       600            1M     2          dest         
0x00000000 149749788  dugi       600          512K     2          dest         
0x00000000 149848094  dugi       600           64M     2          dest         
0x00000000 149815327  dugi       600            1M     2          dest         
0x00000000 166363168  dugi       700          2,2M     2          dest         
0x00000000 166395937  dugi       600          384K     2          dest         
0x00000000 166428706  dugi       600          512K     2          dest         
0x00000000 166461475  dugi       700         53,5K     2          dest         
0x00000000 166625316  dugi       700        376,6K     2          dest         
0x00000000 166658088  dugi       700         13,5K     2          dest         
0x00000000 166690857  dugi       700         88,7K     2          dest         

------ Semaphore Arrays --------
key        semid      owner      perms      nsems     

If I use ipcs -m | awk '{sum+=$5} END{print sum}', it says there's 366790104 bytes, which is 366 MB. This is far less than what it occupies.

I am using Ubuntu 16.04 upgraded to 17.04. The problem was there already before the upgrade. Programs I use almost constantly are QtCreator, Vivaldi, evince, gcc, and the usual ones like gnome shell, nautilus or gedit.

The memory is never swapped out, I can't google a way to find the cause or just to delete it once per day or force swap it or something, I don't want to restart my computer several times per week. Any ideas what can be the problem?

EDIT: I have seen it suck up about 120 megabytes of memory for no known reason. I had Vivaldi (that makes extensive use of shared memory) shut down at that time because I was suspecting it of using causing the problem. The only thing I did at that time is that opened up Firefox for the first time after last restart (Firefox alone cannot be the cause because I didn't run it before those gigabytes got filled).

EDIT #2: Restarting the X-server removes the stuff from shared memory, but that isn't a solution because the X-server is exactly the thing I don't want to restart (gnome-shell --replace doesn't help). If I remember the lessons about shared memory correctly, this means that the memory is grabbed by something related to the X-server or Gnome-shell that correctly frees it on exit, but never erases it while running (some sort of broken garbage collection?).

Edit #3: I wonder if it's related this: https://github.com/mpv-player/mpv/issues/3931

  • When you use gcc, are you developing and compiling your own software? Could there be memory problems with any of the programs that you are compiling? – sudodus Dec 31 '17 at 22:23
  • I am not using shared memory in any program. There may be regular memory leaks, but that's cleared when the program exits. – Dugi Dec 31 '17 at 22:41
2

Maybe smemstat will show the memory hogger(s). Use:

sudo apt-get install smemstat
sudo smemstat
  • After a restart, there isn't enough leaked shared memory yet, so I can't state conclusively if smemstat helped and what it is. The total it shows is for some reasons far greater than the amount of used shared memory announced by free, I think that either some memory is listed multiple times or swapped shared memory is not shown. I will reply after some time to tell if it was the cause. – Dugi Jan 2 '18 at 19:15
  • Okay, so I observed it happening. I had no issues for a long time, but then it happened again. 1456 MiB in shared memory, smemstat lists only unremarkable stuff that amounts to less than 2 gigabytes in total, which is quite different from what free shows. Are you sure smemstat does the trick? Here is what I got: pastebin.com/MhZAWtWU – Dugi Feb 16 '18 at 7:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.