Ubuntu 14.04 on Amazon AWS EC2, has been in production for several years as a web server.
I was working on implementing a new backup process, and while doing so the server ran out of disk space. A 300gb magnetic drive, it's usage was at 290gb when I started this project. While backing up a rather large account, the drive ran out of space but was still mostly operational. OK, so I started deleting files. I deleted a lot of "temp" files local to a few particular web sites, I deleted some working backups (.gz files) that amounted to several gb, and I also checked the /tmp system folder and looked to see if there were any .Trashes folders. I believe in total I deleted at least 10gb of files. However my system still reported zero free space. And MySQL wouldn't start (which I know requires free space to start and operate).
I rebooted the server, to no avail. I shut down apache to make sure it wasn't holding open files. Eventually I expanded the disk drive to 350gb, and then everything started working. The server currently reports 298gb used. Should be much less than that, given all that I deleted.
1) Why would the server not recognize the space I was freeing up? 2) How can I get it to see it now?
Here is the output of df -h
Filesystem Size Used Avail Use% Mounted on
udev 1.8G 12K 1.8G 1% /dev
tmpfs 358M 200K 358M 1% /run
/dev/xvda1 345G 283G 47G 86% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 1.8G 0 1.8G 0% /run/shm
none 100M 0 100M 0% /run/user
overflow 1.0M 76K 948K 8% /tmp
