0

I've got an Ubuntu 12.04.2 server whose root partition filled up. I discovered that the /var/log/auth.log file had grown to around 3.8GB. After compressing this file using gzip, the file size was around 110MB.

Running the df -h command, the root partition's disk usage is still at 100%. I'm assuming a reboot will correct this, but since it's a production server, I was wondering if there was a way to convince this server that it actually has much more free space than it thinks.

Thanks in advance for the help and advice.

EDIT: Lesson learned: lsof is your friend!

After running lsof and grepping for auth.log, I was able to determine that another process was using auth.log. After restarting that process, the free space was recognized by the server.

7
  • Depending on the actual size of the partition these 3.7 GB might not matter much Aug 2, 2013 at 13:05
  • Sorry, I probably should have posted this in the question. It's only an 8 GB partition... we use Amazon Elastic Block Store volumes and attach them to our servers as needed.
    – arochette
    Aug 2, 2013 at 13:14
  • can you post the output of df -i and tune2fs -l /dev/<your root device>? Aug 2, 2013 at 13:19
  • Since I'll be off for a while soon a last hint I can think of: Filesize is not the only thing filling up a hard disk. If you produce enough 0 byte files you can fill it too, because every file uses an inode and a block. Additionally, blocks are reserverd for system use. You can check the usage of both with the commands mentioned above. Aug 2, 2013 at 13:52
  • 1
    Thanks for the advice. Since I don't have 10 reputation, I am not able to answer my own question until 8 hours after posting. Once that time has elapsed, I will answer it in detail and mark it complete.
    – arochette
    Aug 2, 2013 at 14:48

1 Answer 1

0

To figure this out, I ran sudo lsof | grep auth.log. This gave me a list of services that were using the auth.log file.

Rsyslog was one that I had already figured would have auth.log open, so I had already restarted it. What I forgot to take into account was the fact that we use OSSEC, and it had auth.log open as well. Restarting OSSEC solved the problem; the free space was then recognized by the server.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .