2

I have a problem with a server running Ubuntu 16.04.05. After running a python script who created more than 65 million of files I have an error message saying that I have no space left:

12 zip -r 31.zip 31
zip I/O error: No space left on device
zip error: Could not create output file (31.zip)

I found some similar problems online: No space left on the device though there is space and No space left on device even though there is, but I think I have a different one. The solutions posted in these two answers are not working for me.

The following is the structure of the storage:

➜  12 df -k
Filesystem      1K-blocks      Used Available Use% Mounted on
udev             32963768         0  32963768   0% /dev
tmpfs             6597356      9244   6588112   1% /run
/dev/sda1        32895856  26382068   4819736  85% /
tmpfs            32935580        96  32935484   1% /dev/shm
tmpfs                5120         0      5120   0% /run/lock
tmpfs            32935580         0  32935580   0% /sys/fs/cgroup
/dev/sdb       1031992064 664288464 322578204  68% /media/hdd1
tmpfs             6587116        20   6587096   1% /run/user/118
tmpfs             6587116         0   6587116   0% /run/user/1001

I am working and trying to zip a folder in /media/hdd1 which should have plenty of space left. The dimension of the folder I am trying to zip is:

➜  12 du -sh 31
49M     31

EDIT:

➜  12 df -i
Filesystem       Inodes    IUsed   IFree IUse% Mounted on
udev            8240942      478 8240464    1% /dev
tmpfs           8246695      617 8246078    1% /run
/dev/sda1       2097152   500872 1596280   24% /
tmpfs           8233895        5 8233890    1% /dev/shm
tmpfs           8233895        6 8233889    1% /run/lock
tmpfs           8233895       16 8233879    1% /sys/fs/cgroup
/dev/sdb       65536000 65536000       0  100% /media/hdd1
tmpfs           8246695       15 8246680    1% /run/user/118
tmpfs           8246695        4 8246691    1% /run/user/1001
Guido Muscioni
  • 123

2 Answers2

4

"After running a python script who created more than 65 million of files" here is your problem.

The ext4 file system Ubuntu uses has only a limited number of so-called "inodes", which is determined when you format a partition. You need at least one inode per file or directory on the file system.

Now if you create huge numbers of small files, you'll exhaust the available inodes faster than the available storage space. Both equally result in the same symptoms of a seemingly full disk though.

You will need to get rid of those tons of tiny files and e.g. put them on a different drive, in a zip file, or delete them altogether.

Unfortunately the number of inodes can not be changed dynamically, as far as I know. If you really need more on your current partition, you'll probably need to back up your data and reformat everything.

Related:

Byte Commander
  • 110,243
1

As ByteCommander explained, you have a problem with inodes.

Another way to takle this problem is to look for the millions of little files that are consuming the inodes.

sudo du -a -x -d 1 --inodes /media/hdd1 | sort -nr | head -20

will show you the directories consuming the most inodes on /media/hdd1 - you can then travel down through the directories until you find one filled with a bunch of files.

Charles Green
  • 21,859