8

Please see Edit #3 if you're looking for a solution.

I'm running Xubuntu 16.04 LTS, and I have a ~40 GB root partition which is 100% full according to System Monitor (as root). It's definitely full, since many programs aren't functioning correctly.

System Monitor Disk Usage

However, for some reason, I can't seem to figure out what is using the space! Baobab (as root) only reports a total of 15.5 GB used on my root partition!

Baobab Disk Usage

EDIT: Also, here's /var - people said that it's big. Baobab only reports 1 GB for /var, and /var/log is empty. I've tried running sudo rm -R /var/log and there was no effect.

enter image description here

So, how do I find out what is using my disk space, and how do I prevent it from filling up my root partition? This is a huge problem, please help! Thank you in advance :)

EDIT 2: As posted in the answer section, sudo lsof / | awk '{if(\$7 > 1048576) print \$7/1048576 \"MB\" \" \" \$9 }' | sort -n -u returns 11222.7MB /var/log/kern.log 11222.9MB /var/log/syslog, however, I can't seem to figure out how to delete these files, and additionally, I would like to figure out how I can permanently prevent these files from growing this large. This answer to another question suggested that I look into the logs and see what's filling them up, so ideally I'd like some way to read the contents of these mystery files.

EDIT 3: I have temporarily fixed this issue by mounting /var/log on a separate partition.

However, there is still some kind of bug that's causing this. Please, if you want this bug to be fixed, please bring information (or at least give attention) to the bug reports: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1643719 and https://bugzilla.kernel.org/show_bug.cgi?id=188331 Thank you :)

10
  • You could use tail command as in the answer you have mentioned.
    – user.dz
    Nov 21, 2016 at 21:05
  • tail: cannot open '/var/log/syslog' for reading: No such file or directory Nov 21, 2016 at 21:38
  • You need to run it with sudo
    – user.dz
    Nov 21, 2016 at 21:51
  • As root, do cd /proc/$(pidof rsyslogd)/fd ; ls -l | grep /var/log/kern.log. Look at the number to the left of the -> . Run tail -200 thatnumber. Fix the problem (could be a bad piece of hardware). Nov 21, 2016 at 21:52
  • Rebuilding the /var/log hierarchy that you removed will be harder, as there are a few dozen files and directories there that need to have specific owners and permissions. Do you have backups? Nov 21, 2016 at 21:53

4 Answers 4

7

There are two types of file access that use disk space, but don't show up with your tools: deleted (but still open) files, and files being written to.

I have these two aliases defined that I find very useful:

# from http://www.certpal.com/blogs/2010/12/find-open-files-in-linux-using-lsof/
alias bigopenfiles="sudo lsof / | awk '{if(\$7 > 1048576) print \$7/1048576 \"MB\" \" \" \$9 }' | sort -n -u" 

alias deletedfiles="sudo lsof / | egrep 'PID|\(deleted\)'"
9
  • I think you've got something here! bigopenfiles returns 11222.7MB /var/log/kern.log 11222.9MB /var/log/syslog! However, sudo rm /var/log/syslog doesn't work! Nov 21, 2016 at 20:09
  • Restart your system log program and see if the file goes away.
    – Mark
    Nov 21, 2016 at 23:55
  • Read man -k logrotate - it will show you how to keep logs from filling up /. Next time, consider separate partitions for files that can fill up.
    – waltinator
    Nov 22, 2016 at 0:13
  • 1
    rm can remove the name of a file from a directory, but if it's currently open by any process (whether for reading or writing), it'll stay on the disk until the process releases it. Some programs can be sent a signal to tell them to flush, close and re-open their log files; others must be restarted.
    – deltab
    Nov 22, 2016 at 3:51
  • I want to add that I "fixed" this by mounting /var/log on a separate partition. It still constantly fills up :/ Nov 28, 2016 at 21:16
2

Your /var is very fat. Check /var/log

You can use this command to find big files :

sudo find /var -xdev -type f -size +500000k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'

of course you can adjust the size (500000k in this sample)

After that, you can remove or compress them

3
  • Your command returned nothing. I've already tried sudo rm -R /var/log to no effect. I just tried again and it didn't work. I've updated my original post. Nov 21, 2016 at 20:06
  • @f35, can I delete without problem the list I obtained from your find... ? Can I automate the process? Jan 11 at 19:54
  • No you can't ! This command returns the list of big files, you have to check file by file which one is useless
    – f35
    Jan 13 at 7:05
2

You're likely not seeing it with your tools because the file is open. Try this as root

> /var/log/syslog

Exactly, including the >. This will truncate the log.

NOTE:

  • This WILL erase the log!
4
  • bash: /var/log/syslog: No such file or directory Nov 21, 2016 at 20:23
  • And you did that as root? If so, that's really odd and suggests you don't have a syslog...I'm stuck Nov 21, 2016 at 21:20
  • 1
    OP did rm -R /var/log, so the log files only exist as deleted-but-open, and there's nothing in the filesystem to truncate. Nov 21, 2016 at 21:40
  • @Michael Sandman, how do I try that "as root"? Can you be more specific? Jan 11 at 19:55
0

As @waltinator told here if you find your deleted syslog files occupying more space even after deleting them and if you can't free it with rm -R /var/log or > /var/log/syslog, kill the Syslog daemon, (in Elementary OS (Ubuntu 18.04), I did killall rsyslogd to release the open files and freed around 15Gb of / space)

It would be better to mount /var in a separate partition if you've one.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.