A process stopped running. I went to disable the job and can't even access crontab. The instance was provisioned with about 15GB.
$ crontab -e
/tmp/crontab.QdgWmI: No space left on device
Creation of temporary crontab file failed - aborting
When checking the space used there are about 8.5 GB accounted for using du:
sudo du -cha --max-depth=1 / | grep -E "M|G"
139M /boot
791M /snap
2.3G /usr
4.4G /home
287M /run
du: cannot access '/proc/380392/task/380392/fd/4': No such file or directory
du: cannot access '/proc/380392/task/380392/fdinfo/4': No such file or directory
du: cannot access '/proc/380392/fd/3': No such file or directory
du: cannot access '/proc/380392/fdinfo/3': No such file or directory
575M /var
2.2M /tmp
7.2M /etc
8.4G /
8.4G total
df though indicates all 15GB are used.
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 15G 15G 0 100% /
tmpfs 7.9G 0 7.9G 0% /dev/shm
tmpfs 3.2G 295M 2.9G 10% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/xvda16 881M 133M 687M 17% /boot
/dev/xvda15 105M 6.1M 99M 6% /boot/efi
tmpfs 1.6G 12K 1.6G 1% /run/user/1000
This instance should be stateless so I'm not even sure where to start looking.
Targeting different locations with du hasn't uncovered anything that looks exceptional.
