0

Ubuntu 14.04 on Amazon AWS EC2, has been in production for several years as a web server.

I was working on implementing a new backup process, and while doing so the server ran out of disk space. A 300gb magnetic drive, it's usage was at 290gb when I started this project. While backing up a rather large account, the drive ran out of space but was still mostly operational. OK, so I started deleting files. I deleted a lot of "temp" files local to a few particular web sites, I deleted some working backups (.gz files) that amounted to several gb, and I also checked the /tmp system folder and looked to see if there were any .Trashes folders. I believe in total I deleted at least 10gb of files. However my system still reported zero free space. And MySQL wouldn't start (which I know requires free space to start and operate).

I rebooted the server, to no avail. I shut down apache to make sure it wasn't holding open files. Eventually I expanded the disk drive to 350gb, and then everything started working. The server currently reports 298gb used. Should be much less than that, given all that I deleted.

1) Why would the server not recognize the space I was freeing up? 2) How can I get it to see it now?

Here is the output of df -h

Filesystem      Size  Used Avail Use% Mounted on
udev            1.8G   12K  1.8G   1% /dev
tmpfs           358M  200K  358M   1% /run
/dev/xvda1      345G  283G   47G  86% /
none            4.0K     0  4.0K   0% /sys/fs/cgroup
none            5.0M     0  5.0M   0% /run/lock
none            1.8G     0  1.8G   0% /run/shm
none            100M     0  100M   0% /run/user
overflow        1.0M   76K  948K   8% /tmp
  • And while it seemed to be working when I wrote the above, I just now see that I cannot create a cron job because of lack of space. – Rob Brandt May 15 '18 at 17:23
  • Can you edit your question to include the output of df -h please? – muclux May 15 '18 at 17:57
  • After changing the size of the physical disk, did you run resize2fs to resize the filesystem? Read man resize2fs. – waltinator May 16 '18 at 4:14
  • @waltinator, this is an issue that is confusing to me. I have been using Ubuntu on AWS since the 10.04 days, and until I installed a 16.04 server I have NEVER had to resize the file system. It doesn't appear that I need to now either. When I run lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL it shows just one entry, at 350gb. When I run fdisk -l, I get similar data, ending with "Disk /dev/xvda1 doesn't contain a valid partition table". So I guess since my disk isn't split between partitions, I don't have to resize(?). So I guess the reason I don't need to resize is that my disk i – Rob Brandt May 16 '18 at 21:02
  • I believe i have solved the "out of disk space" issue from here askubuntu.com/questions/199565/not-enough-space-on-tmp using option 2 from the approved answer. You can see from my table above that /tmp is mapped to overflow, which apparently is a stopgap Ubuntu does when it runs out of disk space. And then I ran out of overflow space. The solution was to clean out more disk space (down to 280gb used) and reboot. overflow is now gone and it appears /tmp is using regular disk space again. – Rob Brandt May 16 '18 at 21:06

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.