Disk is full but cannot find large files

Question

I was running a bash script which created a 14G file in the tmp directory , I deleted it still I cant find the directory or file which is large.

My output for df -h

Filesystem      Size  Used Avail Use% Mounted on
udev            474M     0  474M   0% /dev
tmpfs            99M   11M   88M  11% /run
/dev/vda1        25G   25G     0 100% /
tmpfs           491M     0  491M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           491M     0  491M   0% /sys/fs/cgroup
/dev/vda15      105M  3.9M  101M   4% /boot/efi
/dev/loop0       90M   90M     0 100% /snap/core/7917
/dev/loop1       55M   55M     0 100% /snap/lxd/12211
/dev/loop2       94M   94M     0 100% /snap/core/8935
/dev/loop3       68M   68M     0 100% /snap/lxd/14194
tmpfs            99M     0   99M   0% /run/user/0
/dev/loop4       55M   55M     0 100% /snap/core18/1705
/dev/loop5       49M   49M     0 100% /snap/gtk-common-themes/1474
/dev/loop6      153M  153M     0 100% /snap/chromium/1071
tmpfs            99M     0   99M   0% /run/user/1000

My output for du -sh in / directory

du: cannot access './proc/19935/task/19935/fd/4': No such file or directory
du: cannot access './proc/19935/task/19935/fdinfo/4': No such file or directory
du: cannot access './proc/19935/fd/3': No such file or directory
du: cannot access './proc/19935/fdinfo/3': No such file or directory
4.7G    .

I cant install ncdu or any other tools since the disk is full , considering the summarized size after du -sh where is the rest of the space out of 25GB

My lsb-release

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=19.10
DISTRIB_CODENAME=eoan
DISTRIB_DESCRIPTION="Ubuntu 19.10"

Output for du -sh /*/

119M    /bin/
97M /boot/
0   /dev/
5.9M    /etc/
212M    /home/
682M    /lib/
4.0K    /lib32/
4.0K    /lib64/
4.0K    /libx32/
16K /lost+found/
4.0K    /media/
4.0K    /mnt/
4.0K    /opt/
du: cannot access '/proc/20791/task/20791/fd/4': No such file or directory
du: cannot access '/proc/20791/task/20791/fdinfo/4': No such file or directory
du: cannot access '/proc/20791/fd/3': No such file or directory
du: cannot access '/proc/20791/fdinfo/3': No such file or directory
0   /proc/
72K /root/
11M /run/
30M /sbin/
1.9G    /snap/
4.0K    /srv/
0   /sys/
17M /tmp/
845M    /usr/
768M    /var/

Answer 1

The POSIX standard allows you to remove a file while it's still being opened. Only when all processes opening the file has closed then the file is actually deleted

The unlink() function shall remove a link to a file. If path names a symbolic link, unlink() shall remove the symbolic link named by path and shall not affect any file or directory named by the contents of the symbolic link. Otherwise, unlink() shall remove the link named by the pathname pointed to by path and shall decrement the link count of the file referenced by the link.

When the file's link count becomes 0 and no process has the file open, the space occupied by the file shall be freed and the file shall no longer be accessible. If one or more processes have the file open when the last link is removed, the link shall be removed before unlink() returns, but the removal of the file contents shall be postponed until all references to the file are closed.

https://pubs.opengroup.org/onlinepubs/009695399/functions/unlink.html

So if the running process in the script is still opening the file the space is still consumed and you can't observe it until the process closes the file or is killed

Answer 2

How about something like:

du -h -t 500M -a /

Usage:

-h, --human-readable
              print sizes in human readable format (e.g., 1K 234M 2G)

 -t, --threshold=SIZE
              exclude  entries  smaller  than  SIZE  if  positive,  or entries
              greater than SIZE if negative

-a, --all
              write counts for all files, not just directories