0

I am seeing some very strange results when using du on some of my folders.

For example:

#cd /storage/main_folder

#du -h --max-depth=2 subfolder1 | grep subfolder2


1.8G    subfolder1/subfolder2/subfolder31
5.3G    subfolder1/subfolder2/subfolder32
352M    subfolder1/subfolder2/subfolder33
76K     subfolder1/subfolder2/subfolder34
1.8G    subfolder1/subfolder2/subfolder35
4.0M    subfolder1/subfolder2/subfolder36
301M    subfolder1/subfolder2/subfolder37
9.4G    subfolder1/subfolder2

# du -h --max-depth=1 subfolder1/subfolder2

1.8G    subfolder1/subfolder2/subfolder31
5.3G    subfolder1/subfolder2/subfolder32
352M    subfolder1/subfolder2/subfolder33
96G     subfolder1/subfolder2/subfolder34
1.8G    subfolder1/subfolder2/subfolder35
4.0M    subfolder1/subfolder2/subfolder36
301M    subfolder1/subfolder2/subfolder37
106G    subfolder1/subfolder2

As far a I can tell, there are no links or special files in subfolder2. The disk is a local RAID volume, ext4, mounted with rw,noatime,nodiratime,stripe=64 .

What could be causing this, and what can I do to fix the problem, and get the correct results?

Thank you!

[ Edit, because I received a link to this question : I am aware of apparent size vs block size. But in this case, note that:

  1. subfolder34 shows up as 76K in one output, and 96G in the other. Too big a difference to be explained by apparent size...
  2. I am using the exact same du command, the only difference is the working directory when running the command. So we can exclude differences due to the way different utilities compute sizes ]
Improve this question
6
  • Hard links, most likely.
    – muru
    Jan 19, 2021 at 9:49
  • How would hard links change the size of the folder? Note that in both examples, I am looking at the exact same folder: /storage/main_folder/subfolder1/subfolder2, yet getting widely different results.
    – Bogd
    Jan 19, 2021 at 13:52
  • 1
    Sure you are, but you're looking at it differently. If I have two folders each with hard links to the same file, each folder's disk usage is the size of the file, but the combined usage of both is also just the size of the file, since the content only exists once.
    – muru
    Jan 19, 2021 at 14:35
  • 1
    @muru - look at subfolder34. It shows up as 76K in the first command output, and 96G in the second one. WAY too big of a difference to be explained by apparent size. As for hard links - even if they were in use, why would du report such a huge difference for the exact same folder? (notice that it is the exact same subfolder34 in both outputs - the only difference is the working dir when running the command). Don't get me wrong - I truly appreciate your comments, and thank you for them! It's just that I cannot wrap my head around this, and this is why I keep asking add'l questions.
    – Bogd
    Jan 19, 2021 at 20:49
  • Did you read the last part of gertvdijk's answer?
    – muru
    Jan 20, 2021 at 1:30

1 Answer 1

Reset to default
1

It looks like @muru hit the nail directly on the head from his first comment - it was just difficult for me to understand how this was happening because of hard links.

du is smart enough to only count hard-linked files once - so in my first example, if it found the files in subfolder34 anywhere else, it wouldn't count them again towards the folder size.

In the second command, since I am only looking at subfolder2, it does not see the other hard links - so it counts the size of the files.

If I run du with -l (telling it explicitly to count hardlinks multiple times), I get the expected results:

#cd /storage/main_folder

#du -h --max-depth=2 subfolder1 | grep subfolder2


1.8G    subfolder1/subfolder2/subfolder31
5.3G    subfolder1/subfolder2/subfolder32
352M    subfolder1/subfolder2/subfolder33
76K     subfolder1/subfolder2/subfolder34
1.8G    subfolder1/subfolder2/subfolder35
4.0M    subfolder1/subfolder2/subfolder36
301M    subfolder1/subfolder2/subfolder37
9.4G    subfolder1/subfolder2

#du -h --max-depth=2 -l subfolder1 | grep subfolder2

1.8G    subfolder1/subfolder2/subfolder31
5.3G    subfolder1/subfolder2/subfolder32
352M    subfolder1/subfolder2/subfolder33
96G     subfolder1/subfolder2/subfolder34
1.8G    subfolder1/subfolder2/subfolder35
4.0M    subfolder1/subfolder2/subfolder36
301M    subfolder1/subfolder2/subfolder37
106G    subfolder1/subfolder2

Thank you, @muru, for the comments, and for your patience with my many questions! :) And if you would post your comment as an answer, I will gladly accept it.

Improve this answer
1
  • It's fine, I think gertvdijk's answer on the dupe, or your answer here, or as it turns out, my own answer elsewhere all address this well enough that I don't need to write an answer here.
    – muru
    Jan 20, 2021 at 11:10

Not the answer you're looking for? Browse other questions tagged .