0

I've got a system that collects a lot of data, than backs it up occasionally and pushes it to another server. My problem is that, while data collection process is taking 2-3% of memory, the rest of RAM fills up with PageCache, which causes problems for me.

sysctl -w vm.drop_caches=1

clears the memory, and I can set up chron job to clean it now and than, but this seems like inelegant solution. I'd rather disable the pagecache altogether on the raid disc that the app uses, or if that is not possible, on the whole server.

I've tried this code to start the app, and it doesn't work for me, pagecache keeps accumulating.

I'm running the program on Ubuntu server 20

How do I disable the PageCache on a hard drive/whole system?

UPDATE

Here's free over time:

>>> free
              total        used        free      shared  buff/cache   available
Mem:        3981820      202180     3600504         852      179136     3565116
Swap:             0           0           0
>>>free
              total        used        free      shared  buff/cache   available
Mem:        3981820      202336     3593204         852      186280     3561364
Swap:             0           0           0
>>>free
              total        used        free      shared  buff/cache   available
Mem:        3981820      203132     3505208         852      273480     3547512
Swap:             0           0           0

Buff/cache continues growing until I flush the memory

>>>sudo lshw -C memory
  *-firmware                
       description: BIOS
       vendor: Amazon EC2
       physical id: 0
       version: 1.0
       date: 10/16/2017
       size: 64KiB
       capacity: 64KiB
       capabilities: pci edd acpi virtualmachine
  *-memory
       description: System memory
       physical id: 1
       size: 4GiB
>>>sysctl vm.swappiness
vm.swappiness = 60
15
  • Where/how are you seeing the pagecache problem?
    – heynnema
    Aug 25, 2020 at 16:11
  • @heynnema I'm starting to get memory exceptions in the app around the time free system memory is almost empty Aug 25, 2020 at 16:13
  • Edit your question and show me free -h and sudo lshw -C memory and sysctl vm.swappiness.
    – heynnema
    Aug 25, 2020 at 16:14
  • And reset your disk write-cache.
    – heynnema
    Aug 25, 2020 at 16:15
  • @heynnema updated. and how do I reset write-cache? Aug 25, 2020 at 16:25

3 Answers 3

2

You have NO swap!

$ free

              total        used        free      shared  buff/cache   available
Mem:        3981820      202180     3600504         852      179136     3565116
Swap:             0           0           0

Let's create a /swapfile...

Note: Incorrect use of the dd command can cause data loss. Suggest copy/paste.

sudo swapoff -a           # turn off swap
sudo rm -i /swapfile      # remove old /swapfile

sudo dd if=/dev/zero of=/swapfile bs=1M count=4096

sudo chmod 600 /swapfile  # set proper file protections
sudo mkswap /swapfile     # init /swapfile
sudo swapon /swapfile     # turn on swap
free -h                   # confirm 4G RAM and 4G swap

Confirm this line in /etc/fstab... and confirm no other “swap” lines...

/swapfile    none    swap    sw      0   0

reboot                    # reboot and verify operation

Update #1:

I did some investigation into issue and found that the problem I was having when I was approaching the limit of free memory actually was that the program was hitting open file limit, so I ended up fixing that. Adding/removing swap didn't make any difference, neither did running a cronjob that would clean the memory.

0

Install the nocache package:

sudo apt-get install nocache

Then run your process with the nocache command:

nocache whatever

This should control the cache build-up problem at the source.

0

To disable the write cache, try this:

Edit /etc/hdparm.conf.

Uncomment the second of these two lines so that it looks like this:

# -W Disable/enable the IDE drive's write-caching feature
write_cache = off

To enable the write cache:

Edit /etc/hdparm.conf.

Comment the second of these two lines so that it looks like this:

# -W Disable/enable the IDE drive's write-caching feature
# write_cache = off
3
  • what is the command to re-load configuration? Aug 25, 2020 at 15:43
  • I've restarted the server with the new setting, and it still is having this issue Aug 25, 2020 at 16:12
  • I don't think that disk write-cache is what OP is asking about.
    – heynnema
    Aug 25, 2020 at 16:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .