I am aware of buffers, cache, swap, etc. but can anyone explain this one to me?
First, Here's a TOP output sorted by memory usage. Note that MySQL is using 39.4% of the RAM in the system. Every other process is using so little it reports as zero:
Tasks: 728 total, 2 running, 726 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.8 us, 1.8 sy, 0.0 ni, 97.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 225344.3 total, 4238.5 free, 217754.1 used, 3351.8 buff/cache
MiB Swap: 8192.0 total, 8176.0 free, 16.0 used. 5988.2 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2229271 mysql 20 0 98.9g 86.8g 39456 S 14.9 39.4 623:34.91 mysqld
45955 grafana+ 20 0 4302196 74996 26664 S 2.6 0.0 194:44.43 grafana-agent
45353 root 20 0 3165840 48080 20172 S 0.0 0.0 1:06.33 snapd
834 root 19 -1 117880 45432 43988 S 0.0 0.0 0:04.42 systemd-journal
45494 root 20 0 111680 36748 32476 S 0.3 0.0 0:05.86 sssd_nss
45490 root 20 0 142632 35488 22776 S 0.0 0.0 0:13.36 sssd_be
45495 root 20 0 86644 23020 19040 S 0.0 0.0 0:03.42 sssd_sudo
etc...
Next look at my free -g report:
# free -g
total used free shared buff/cache available
Mem: 220 212 4 0 3 5
Swap: 7 0 7
If MySQL is only using 39.4% of the 220 GB (86.68 GB,) and cache is taking 3 GB, where's the other 130GB going? If I restart the MySQL service the total system memory in use will drop to about 12 GB then slowly climbs and climbs. Usually MySQL will report as using 90% of the RAM but this time when I looked I saw the reports you see above.
