$ cat /etc/fstab
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
UUID=a168d1ac-4e13-4643-976d-6e47ea1732b1 /boot        ext2  defaults                                                                   0 1
/dev/mapper/sda4_crypt                    /            btrfs defaults,autodefrag,compress=lzo,inode_cache,space_cache,subvol=@          0 2
/dev/mapper/sda4_crypt                    /tmp         btrfs defaults,autodefrag,compress=lzo,inode_cache,space_cache,subvol=@tmp       0 2
/dev/mapper/sda4_crypt                    /run         btrfs defaults,autodefrag,compress=lzo,inode_cache,space_cache,subvol=@run       0 2
/dev/mapper/sda4_crypt                    /var/crash   btrfs defaults,autodefrag,compress=lzo,inode_cache,space_cache,subvol=@var-crash 0 2
/dev/mapper/sda4_crypt                    /var/tmp     btrfs defaults,autodefrag,compress=lzo,inode_cache,space_cache,subvol=@var-tmp   0 2
/dev/mapper/sda4_crypt                    /var/log     btrfs defaults,autodefrag,compress=lzo,inode_cache,space_cache,subvol=@var-log   0 2
/dev/mapper/sda4_crypt                    /var/spool   btrfs defaults,autodefrag,compress=lzo,inode_cache,space_cache,subvol=@var-spool 0 2
/dev/mapper/sda5_crypt                    /home        btrfs defaults,autodefrag,compress=lzo,inode_cache,space_cache,subvol=@home      0 3
/dev/mapper/750er                         /media/750er ext4  defaults                                                                   0 4
/dev/mapper/cswap                         none         swap  defaults                                                                   0 5
➜  ~  df -h         
Filesystem              Size  Used Avail Use% Mounted on
/dev/mapper/sda4_crypt   38G   12G   13M 100% /
none                    4,0K     0  4,0K   0% /sys/fs/cgroup
udev                    2,0G  4,0K  2,0G   1% /dev
tmpfs                   396M  1,3M  394M   1% /run
none                    5,0M     0  5,0M   0% /run/lock
none                    2,0G  208K  2,0G   1% /run/shm
none                    100M   36K  100M   1% /run/user
/dev/mapper/sda4_crypt   38G   12G   13M 100% /tmp
/dev/sda2               231M   44M  175M  21% /boot
/dev/mapper/sda4_crypt   38G   12G   13M 100% /var/crash
/dev/mapper/sda4_crypt   38G   12G   13M 100% /var/tmp
/dev/mapper/sda4_crypt   38G   12G   13M 100% /var/log
/dev/mapper/sda4_crypt   38G   12G   13M 100% /var/spool
/dev/mapper/sda5_crypt  3,7T  2,4T  1,2T  67% /home
/dev/mapper/750er       688G  276G  377G  43% /media/750er
/dev/mapper/2tb         1,8T  1,7T  141G  93% /media/2tb
➜  ~  sudo btrfs fi df /
Data, single: total=9.47GiB, used=9.46GiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=13.88GiB, used=1.13GiB
Metadata, single: total=8.00MiB, used=0.00
➜  ~  

It's a 40GB partition with quite some snapshots on it. But it's compressed so I think the 9,46GB/40GB is accurate. But my Ubuntu fails because it says it has no disk space. I had apt-errors, cant install programs and my mysql server failed to start because of it.

And I know about not to rely on df I just included it for completeness.

I think Ubuntu uses df which is known to report wrongly with Btrfs internally and fails for that reason. That would make sense for APT when it does checks for space. But it actually fails to write to disk.

$ sudo time dd if=/dev/zero of=large bs=2G count=1
dd: error writing ‘large’: No space left on device
0+1 records in
0+0 records out
11747328 bytes (12 MB) copied, 1,29706 s, 9,1 MB/s
Command exited with non-zero status 1
0.00user 1.40system 0:01.44elapsed 97%CPU (0avgtext+0avgdata 2098028maxresident)k
160inputs+23104outputs (0major+383008minor)pagefaults 0swaps
  • 2
    Duplicate of btrfs and missing free space – bain May 10 '14 at 21:40
  • 1
    How does the answer in the duplicate I linked to (explaining btrfs fi df) not answer your question? – bain May 10 '14 at 21:44
  • I do not understand the statement "The difference between total and used (around 15 GB) is your missing space. As you can see, it has already been allocated for metadata blocks." All I know is that I have a 40 Gig partiton and it shows me used=9.46GiB. Do I need to add the 13.88GiB DUP Metadata (whatever this means) on top of that? Even if there is still lots of free space. To me it seems this guy has indeed a full disk while for me this is not the case. – redanimalwar May 10 '14 at 21:51
  • As a temporary workaround, you may add a device to the btrfs filesystem, and remove it later, when you have solved the problem. – ignis May 11 '14 at 7:54
  • 2
    Side note: you should have 0 0 in your fstab for the btrfs subvolumes. See this FAQ – ignis May 11 '14 at 8:02
up vote 15 down vote accepted

Btrfs is different from traditional filesystems. It is not just a layer that translates filenames into offsets on a block device, it is more of a layer that combines a traditional filesystem with LVM and RAID. And like LVM, it has the concept of allocating space on the underlying device, but not actually using it for files.

A traditional filesystem is divided into files and free space. It is easy to calculate how much space is used or free:

|--------files--------|                                                |
|------------------------drive partition-------------------------------|

Btrfs combines LVM, RAID and a filesystem. The drive is divided into subvolumes, each dynamically sized and replicated:

|--files--|    |--files--|         |files|         |                   |
|----@raid1----|------@raid1-------|-----@home-----|metadata|          |
|------------------------drive partition-------------------------------|

The diagram shows the partition being divided into two subvolumes and metadata. One of the subvolumes is duplicated (RAID1), so there are two copies of every file on the device. Now we not only have the concept of how much space is free at the filesystem layer, but also how much space is free at the block layer (drive partition) below it. Space is also taken up by metadata.

When considering free space in Btrfs, we have to clarify which free space we are talking about - the block layer, or the file layer? At the block layer, data is allocated in 1GB chunks, so the values are quite coarse, and might not bear any relation to the amount of space that the user can actually use. At the file layer, it is impossible to report the amount of free space because the amount of space depends on how it is used. In the above example, a file stored on the replicated subvolume @raid1 will take up twice as much space as the same file stored on the @home subvolume. Snapshots only store copies of files that have been subsequently modified. There is no longer a 1-1 mapping between a file as the user sees it, and a file as stored on the drive.

You can check the free space at the block layer with btrfs filesystem show / and the free space at the subvolume layer with btrfs filesystem df /


# df -h
Filesystem              Size  Used Avail Use% Mounted on
/dev/mapper/sda4_crypt   38G   12G   13M 100% /

For this mounted subvolume, df reports a drive of total size 38G, with 12G used, and 13M free. 100% of the available space has been used. Remember that the total size 38G is divided between different subvolumes and metadata - it is not exclusive to this subvolume.

# btrfs filesystem df /
Data, single: total=9.47GiB, used=9.46GiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=13.88GiB, used=1.13GiB
Metadata, single: total=8.00MiB, used=0.00

Each line shows the total space and the used space for a different data type and replication type. The values shown are data stored rather than raw bytes on the drive, so if you're using RAID-1 or RAID-10 subvolumes, the amount of raw storage used is double the values you can see here.

The first column shows the type of item being stored (Data, System, Metadata). The second column shows whether a single copy of each item is stored (single), or whether two copies of each item are stored (DUP). Two copies are used for sensitive data, so there is a backup if one copy is corrupted. For DUP lines, the used value has to be doubled to get the amount of space used on the actual drive (because btrfs fs df reports data stored, not drive space used). The third and fourth columns show the total and used space. There is no free column, since the amount of "free space" is dependent on how it is used.

The thing that stands out about this drive is that you have 9.47GiB of space allocated for ordinary files of which you have used 9.46GiB - this is why you are getting No space left on device errors. You have 13.88GiB of space allocated for duplicated metadata, of which you have used 1.13GiB. Since this metadata is DUP duplicated, it means that 27.76GiB of space has been allocated on the actual drive, of which you have used 2.26GiB. Hence 25.5GiB of the drive is not being used, but at the same time is not available for files to be stored in. This is the "Btrfs huge metadata allocated" problem. To try and correct this, run btrfs balance start -m /. The -m parameter tells btrfs to only re-balance metadata.

A similar problem is running out of metadata space. If the output had shown that the metadata were actually full (used value close to total), then the solution would be to try and free up almost empty (<5% used) data blocks using the command btrfs balance start -dusage=5 /. These free blocks could then be reused to store metadata.

For more details see the Btrfs FAQs:

  • Note that, currently, btrfs cannot have different RAID on different subvolumes. However this is planned. – ignis May 11 '14 at 7:37
  • To balance only the metadata, use btrfs filesystem balance -m, otherwise the data will also be rewritten (which is useless in this case) – ignis May 11 '14 at 7:49
  • Also note that actually 27 GiB are allocated for metadata (source - "The values shown are data rather than raw bytes, so if you're using RAID-1 or RAID-10, the amount of raw storage used is double the values you can see here.") – ignis May 11 '14 at 7:50
  • @ignis Thank you for your comments, I have edited the answer. – bain May 11 '14 at 11:19
  • 1
    @redanimalwar The answer has an example and explanation of free space in btrfs because you said you didn't understand the explanation at btrfs and missing free space. The problem there is exactly the same problem as here - check the btrfs fi df output - he had 49GB of a 75GB drive allocated to metadata, with 29GB of that not being used. – bain May 11 '14 at 22:40

Short answer: Btrfs partition metadata is shown as 'used' by Standard disk utilities like df.

  1. check the problem volume. for example: /

    btrfs subvolume list /
    
  2. Most likely snapshots are filling the volume. Delete snapshot you do not need. keep one from last date you are sure system was working ok.

    btrfs subvolume delete <path> 
    

    Where path is from previous command subvolume list that says "snapshot".

  3. Reboot and you are done

Reason for problem might be your distro or package manager making snaphots every time you update system.

NB: balance command fails if disk is full as there is no free space to blance to.

In my case the disk usage would not go down even when I deleted files and snapshots.

btrfs balance (data and metadata) did not work with error "no space left on device"

btrfs balance start -m /
ERROR: error during balancing '/': No space left on device
There may be more info in syslog - try dmesg | tail

The RAID1 showed full usage on both disks even though actual data usage was lower than a third of that.

# btrfs fi sh
Label: none  uuid: 61a20f1a-c133-11e6-964b-d3bac0c48bbd
    Total devices 2 FS bytes used 153.94GiB
    devid    1 size 455.76GiB used 455.76GiB path /dev/sda2
    devid    2 size 455.76GiB used 455.76GiB path /dev/sdb2


# btrfs filesystem df /
Data, RAID1: total=452.73GiB, used=151.51GiB
System, RAID1: total=32.00MiB, used=80.00KiB
Metadata, RAID1: total=3.00GiB, used=2.42GiB
GlobalReserve, single: total=512.00MiB, used=0.00B

Solution: discard empty chunks, needs no additional space:

btrfs balance start -dusage=0 /

btrfs balance start -musage=0 /

Source: https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-balance#ENOSPC

Alternative: My solution was to shrink the disks See: https://unix.stackexchange.com/questions/239765/how-to-fix-btrfs-superblock-error-after-resize-shrink-btrfs-couldnt-get-super

btrfs filesystem resize 1:430g /
btrfs filesystem resize 2:430g /

(commands take time, check syslog to see relocating blocks)

after that resize up:

btrfs filesystem resize 1:450g /
btrfs filesystem resize 2:450g /

After that, btrfs balance (metadata) worked again:

btrfs balance -m /

Then btrfs balance of the data (relocate data-chunks that have lower than 33% usage):

btrfs balance -dusage=33 /

Credit goes to @ignis and @bain. Just to have a simple straight to the point reference answer here without all the lecture and to share what I actually did to get system working again.

btrfs balance start -m /mountpoint

Is the magic line to solve issues like this.

I ran into some trouble that I don't want to bore you with and I don't know it was necessary to run this from a live cd but what I did in the end after the system was messed up not booting was running btrfsck on the devices (unlocked crypto mappers) and it did actually found errors, then mounted the root btrfs filesystem without any options to /mnt, as opposed to on my installed system where / is the only the mounted @ subvolume. So I had all the snapshots and other subvolumes also in there. I do not know of this makes a difference.

btrfs balance start -m /mnt

I got the metadata from

Metadata, DUP: total=13.88GiB, used=1.13GiB

to

Metadata, DUP: total=1.38GiB, used=1.05GiB

And a refreshing:

$ sudo df -h /
Filesystem              Size  Used Avail Use% Mounted on
/dev/mapper/sda4_crypt   38G   12G   26G  31% /

So I think everything is fine now.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.