Disk usage increasing over time. Why? And how to fix?

Question

I have an Ubuntu server VPS with DigitalOcean. This Ubuntu server contains a very simple web server and website setup, and not much else. However, whenever I create an image of the droplet (using the DigitalOcean control panel) the size is always increasing. The last image is 13GB, up from 9GB a few weeks ago... 7GB ... 5GB... just a few months ago.

So the disk space is getting eaten up, and I'm not sure why.

The /var/ directory where the website and other related assets are, including all the log files, is only using 1.3GB. So apparently this is not the source of the problem.

1. how can i determine what is eating up this disk space? Yes, I known I can use du, etc., but the problem is I don't know what would normally and legitimately be taking up a lot of space and what wouldn't. I want to see where this expansion in disk usage is occurring so I can then investigate to see if I can purge old/unnecessary files.

2. can you tell me what directories/files on an ubuntu server typically need pruning over time to prevent wasted disk space like I am experiencing?

3. I have enabled automatic system security updates. Could this be contributing to the problem? How would I check this? Can I clean out old files? Where would they be?

Answer 1

First, how can i determine what is eating up this disk space?

find . -type f  -exec du -h {} + | sort -r -h > /tmp/files.txt

to list files ordered by size.

but the problem is I don't know what would normally and legitimately be taking up a lot of space and what wouldn't.

Neither do we. What you can do is create a list of all files with the command above and compare it to the same list next week. Find the files that got bigger or the new files (diff is a good tool for that).

Second, can you tell me what directories/files on an ubuntu server typically need pruning over time to prevent wasted disk space like I am experiencing?

• did you install software? Ubuntu keeps a cache of packages. But generally this will not be the issue: a server has its software and when working should not have new software installed. Only updates and those are small (the mostly replace the existing software).

Third, I have enabled automatic system security updates. Could this be contributing to the problem? How would I check this? Can I clean out old files? Where would they be?

• No
• No.
• That's up to you to decide.
• see for instance var/cache/apt/archives occupying huge space for deleting cache archived packages.

Answer 2

There are a few possible reasons why your disk usage is increasing over time:

1. Log files: The log files for your web server and other services can grow over time and take up a significant amount of disk space.

2. Automatic updates: Enabling automatic system security updates can cause the disk usage to increase over time if the old update files are not being cleaned up. Ubuntu stores old versions of packages in the /var/cache/apt/archives/ directory, so if this directory is not being cleaned up, it can grow over time.

3. Data storage: If your web app is storing user data or other files on the server, this could also be contributing to the disk usage growth.

In terms of directories that typically need pruning over time on an Ubuntu server, here are a few examples:

1. The /var/log/ directory: This directory contains log files for various services. You can use the logrotate tool to automatically rotate and prune log files on a regular basis.

2. The /var/cache/apt/archives/ directory: This directory stores packages that have been downloaded by the Ubuntu package manager. You can clean up old packages using the apt-get autoclean command.

3. The /tmp/ directory: This directory contains temporary files created by various applications. You can use the tmpreaper tool to automatically clean up old files in this directory.

To check if automatic system security updates are contributing to the problem, you can check the /var/log/apt/ directory for logs related to package updates.

You can also check the /var/log/unattended-upgrades/ directory for logs related to automatic security updates.

To clean up old update files, you can use the apt-get autoremove command.