0

I am seeing some very strange results when using du on some of my folders.

For example:

#cd /storage/main_folder

#du -h --max-depth=2 subfolder1 | grep subfolder2


1.8G    subfolder1/subfolder2/subfolder31
5.3G    subfolder1/subfolder2/subfolder32
352M    subfolder1/subfolder2/subfolder33
76K     subfolder1/subfolder2/subfolder34
1.8G    subfolder1/subfolder2/subfolder35
4.0M    subfolder1/subfolder2/subfolder36
301M    subfolder1/subfolder2/subfolder37
9.4G    subfolder1/subfolder2


du -h --max-depth=1 subfolder1/subfolder2


1.8G    subfolder1/subfolder2/subfolder31
5.3G    subfolder1/subfolder2/subfolder32
352M    subfolder1/subfolder2/subfolder33
96G     subfolder1/subfolder2/subfolder34
1.8G    subfolder1/subfolder2/subfolder35
4.0M    subfolder1/subfolder2/subfolder36
301M    subfolder1/subfolder2/subfolder37
106G    subfolder1/subfolder2

As far a I can tell, there are no links or special files in subfolder2. The disk is a local RAID volume, ext4, mounted with rw,noatime,nodiratime,stripe=64 .

What could be causing this, and what can I do to fix the problem, and get the correct results?

Thank you!

[ Edit, because I received a link to this question : I am aware of apparent size vs block size. But in this case, note that:

  1. subfolder34 shows up as 76K in one output, and 96G in the other. Too big a difference to be explained by apparent size...
  2. I am using the exact same du command, the only difference is the working directory when running the command. So we can exclude differences due to the way different utilities compute sizes ]
Bogd
  • 189

1 Answers1

2

It looks like @muru hit the nail directly on the head from his first comment - it was just difficult for me to understand how this was happening because of hard links.

du is smart enough to only count hard-linked files once - so in my first example, if it found the files in subfolder34 anywhere else, it wouldn't count them again towards the folder size.

In the second command, since I am only looking at subfolder2, it does not see the other hard links - so it counts the size of the files.

If I run du with -l (telling it explicitly to count hardlinks multiple times), I get the expected results:

#cd /storage/main_folder

#du -h --max-depth=2 subfolder1 | grep subfolder2


1.8G    subfolder1/subfolder2/subfolder31
5.3G    subfolder1/subfolder2/subfolder32
352M    subfolder1/subfolder2/subfolder33
76K     subfolder1/subfolder2/subfolder34
1.8G    subfolder1/subfolder2/subfolder35
4.0M    subfolder1/subfolder2/subfolder36
301M    subfolder1/subfolder2/subfolder37
9.4G    subfolder1/subfolder2


#du -h --max-depth=2 -l subfolder1 | grep subfolder2


1.8G    subfolder1/subfolder2/subfolder31
5.3G    subfolder1/subfolder2/subfolder32
352M    subfolder1/subfolder2/subfolder33
96G     subfolder1/subfolder2/subfolder34
1.8G    subfolder1/subfolder2/subfolder35
4.0M    subfolder1/subfolder2/subfolder36
301M    subfolder1/subfolder2/subfolder37
106G    subfolder1/subfolder2

Thank you, @muru, for the comments, and for your patience with my many questions! :) And if you would post your comment as an answer, I will gladly accept it.

Bogd
  • 189