"No space left on device" when installing packages, yet I have space left

Question

Whether through update manager, or through terminal, I error out.

When I try sudo apt-get update && sudo apt-get upgrade:

Fetched 1,024 kB in 20s (51.0 kB/s) 
Reading package lists... Done 
Reading package lists... Done 
Building dependency tree Reading state information... Done 
You might want to run 'apt-get -f install' to correct these. 
The following packages have unmet dependencies: linux-headers-generic : 
Depends: linux-headers-3.13.0-29-generic but it is not installed 
E: Unmet dependencies. Try using -f.

And When I try sudo apt-get -f install:

Unpacking linux-headers-3.13.0-29-generic (3.13.0-29.53) ... 
dpkg: error processing archive /var/cache/apt/archives/linux-headers-3.13.0-29-generic_3.13.0-29.53_amd64.deb 
(--unpack): unable to create /usr/src/linux-headers-3.13.0-29-generic/include/config/inotify/user.h.dpkg-new' (while processing./usr/src/linux-headers-3.13.0-29-generic/include/config/inotify/user.h'): No space left on device 
No apport report written because the error message indicates a disk full error dpkg-deb: error: subprocess paste was killed by signal (Broken pipe) 
Errors were encountered while processing: /var/cache/apt/archives/linux-headers-3.13.0-29-generic_3.13.0-29.53_amd64.deb 
E: Sub-process /usr/bin/dpkg returned an error code (1)

When I open Ubuntu software center, I get the following text:

New software can't be installed because there is a problem with the software currently installed. Do you want to repair this problem now?

I get the option to 'repair' or 'cancel'. If I hit 'repair', I get the following error:

Package operation failed
the installation or removal of a software package failed.
(Reading database ... 417598 files and directories currently installed.) 
Preparing to unpack .../linux-headers-3.13.0-29-generic_3.13.0-29.53_amd64.deb ...   Unpacking linux-headers-3.13.0-29-generic (3.13.0-29.53) ... 
dpkg: error processing archive /var/cache/apt/archives/linux-headers-3.13.0-29-generic_3.13.0-29.53_amd64.deb 
(--unpack): unable to create /usr/src/linux-headers-3.13.0-29-generic/include/config/bug.h.dpkg-new' (while processing./usr/src/linux-headers-3.13.0-29-generic/include/config/bug.h'): No space left on device 
No apport report written because the error message indicates a disk full error dpkg-deb: error: subprocess paste was killed by signal (Broken pipe) 
Errors were encountered while processing: /var/cache/apt/archives/linux-headers-3.13.0-29-generic_3.13.0-29.53_amd64.deb 
Error in function: dpkg: dependency problems prevent configuration of linux-headers-generic: linux-headers-generic depends on linux-headers-3.13.0-29-generic; however: Package linux-headers-3.13.0-29-generic is not installed.
dpkg: error processing package linux-headers-generic (--configure): dependency problems - leaving unconfigured 
dpkg: dependency problems prevent configuration of linux-generic: linux-generic depends on linux-headers-generic (= 3.13.0.29.35); however: Package linux-headers-generic is not configured yet.
dpkg: error processing package linux-generic (--configure): dependency problems - leaving unconfigured

The disk full error is odd, because none of my disks are full. I know this because when I run df, I get the following:

Filesystem 1K-blocks      Used Available Use% Mounted on 
/dev/sda6   36178648  13162504  21155332  39% /
none               4         0         4   0% /sys/fs/cgroup
udev         3873544        12   3873532   1% /dev
tmpfs         776860      1384    775476   1% /run
none            5120         0      5120   0% /run/lock
none         3884300     25276   3859024   1% /run/shm
none          102400        68    102332   1% /run/user
/dev/sda2  124327968  42583252  81744716  35% /media/sudoer/48AEC994AEC97B48 
/dev/sda3  807468724 121812996 685655728  16% /media/sudoer/New Volume

What do I do at this point? Thanks in advance.

Answer 1

On unix file system, file and directory have two part: inode and data.

In inodes are stored information about file such as permission, owner, change time, access time, and so on.

The total number of inodes is limited, so you can get No space left on device if you reach such limit even if the disk is not very full.

Use:

df -i

to view inodes, if you get 100% for IUse and 0 IFree, this means that you have a lot of small file.

For example if you have 2 billions files whose size is 1 byte, you could reach inode limit, but your disk could be not completely full.

If this is your case, try to remove more file you can and than redo apt-get update.

Answer 2

You can clean up space in /boot (remove old linux headers etc), with:

sudo apt-get autoremove

Answer 3

I also encountered this problem. According to the prompt,

unable to create /usr/src/linux-headers-3.13.0-29-generic/include/...

As explained by Lety, "The total number of inodes is limited". So I deleted some old linux-headers packages using the package manager so /usr will have enough inodes, for example sudo apt-get remove linux-headers-3.13.0-3*, then I can run sudo apt-get install -f to continue the installation.

Answer 4

Your tmpfs may be full. /tmp lives in your RAM or swap. You can check how much space is being used by running:

$ df -h | grep tmpfs
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           3.9G   33M  3.9G   1% /dev/shm
tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
tmpfs           3.9G  3.9G     0 100% /tmp

To temporarily increase its size, try the following command:

sudo mount -o remount,size=16G,noatime /tmp

You can also permanently increase the size through /etc/fstab.

Answer 5

As @Lety said, the problem is how to find folders with a lot of tiny files which wasted all your inodes (as df -i show).

Find heavy packages (like old kernels) with

wajig sizes | tail -n 30

which requiere wajig installed, otherwise:

dpkg-query --show --showformat='${Package;-50}\t${Installed-Size}\n' | sort -k 2 -n | grep -v deinstall | awk '{printf "%.3f MB \t %s\n", $2/(1024), $1}'|tail -n 10

Use uname -r to find current kernel, you don't want to remove it.

Then try:

1. Reboot or try if there is any process to kill which opened a large file which has since been deleted sudo lsof +L1
2. Remove unused packages sudo apt-get autoremove
3. Clean cache sudo apt-get clean or manually on /var/cache/apt/archives
4. Use apt-get remove or sudo dpkg --purge to remove some old kernels or heavy packages. Maybe you still get unable to create /usr/src/linux-headers...
5. If you get here you will use those slow techniques to find the guilty folder, but first go to the usual suspects using something like for i in /usr/src/*; do echo $i; find $i |wc -l; done:
   1. Linux header folders: /usr/src/linux-headers-*
   2. Logs: /var/log/

Specific folder counters:

• du --inodes -d 1 /usr/src/ | sort -n
• sudo find . -xdev -type f | cut -d "/" -f 2 | sort | uniq -c | sort -n
• sudo find . -maxdepth 1 -type d | grep -v '^\.$' | xargs -n 1 -i{} find {} -xdev -type f | cut -d "/" -f 2 | uniq -c | sort -n

Answer 6

I had the problem that there was no space and no inodes on my /boot partition left. I tried to remove old kernels via dpkg but it was not doing the trick. After inspecting the folder with du -h and ls -h I found that most space was used by initramfs files.

To uninstall them I had to follow this guide: https://help.ubuntu.com/community/RemoveOldKernels

• First I checked which kernel version I was using via uname -r
• Then I found other installed kernel versions via dpkg -l | grep linux-image | grep ii (the grep ii ensures that only installed versions are listed)
• I choose the oldest kernels and used the command sudo update-initramfs -d -k 4.2.0-15-generic. I don't know if that is strictly neccessary.
• Then I purged the kernel with dpkg --purge 4.2.0-15-generic
• Now I was finally able to run apt -f install
• and sudo apt autoremove

Answer 7

Running out of inodes on Ubuntu: if below commands showing the same error go to /usr/src and delete old kernel(unused) directories so it will resolve the issues.

sudo apt-get -f install
sudo apt autoremove

For more information, see: https://medium.com/@brucepomeroy/running-out-of-inodes-on-ubuntu-718aef71ac16