I have a directory on Ubuntu that contains 262144 files. Each file is a 25x25 png image. On average these files are approximately 1.14kb and no file is larger than 2kb. Yet this directory is using up 3.1GB of disk space. How is this possible? By my calculations this directory should be using 262144 * 1140 = 298844160 bytes
, which is only 0.298844 GB
.
Here are the steps I followed to get this information.
I ran ls -1 -f | wc -l
to count the number of files in the directory. This returns 262146
(i.e., 262144 + 1 + 1
for .
and ..
).
Next I ran find . -size +2k
and the result was just .
.
Finally I ran du -sh culprit_directory
and the result shows 3.1G culprit_directory
.
There are two things that I imagine could be happening:
- Ubuntu needs extra space to store a directory that contains a very large number of very small files. Possible, but a whole order of magnitude?
- I am making a mistake in my calculations and this is the expected size for the directory. Also possible, but I am unable to see where I made this mistake.
If anyone with more experience with Ubuntu's internal file storage could advise me I would greatly appreciate it.
EDIT:
I have added one of the png files. This one is 591 bytes
in size.
EDIT:
Thanks to muru's helpful comments below, I have determined that each file is actually using 12KB
on disk, even if it only consists of a few hundred bytes. Using the new numbers, we get 262144 * 12000 = 3145728000
, which gives us 3.145728GB
.
I guess my new question would be how to avoid each file using so much space?
ls -A | wc -l
, anddu -h --max-depth 1 culprit-directory | sort -h | tail
.du -h
should use GB and not Gb.4.0K
, that still does not explain the10x
size increase. When I run that command I getbash: /usr/bin/du: Argument list too long
again :(sudo dumpe2fs /dev/sdaX | grep 'Block size'
- That's assuming you have an ext4 filesystem./dev/sdaX
is the partition which contains culprit-directory.