This answer and email message indicate that something called "OverlayFS" is available in Ubuntu 11.10 and will forcefully replace aufs in Ubuntu 12.04.

How do I use it? Where is its documentation?

  • 1
    You and me both, brother. I've come across this so far: mount -t overlayfs -o rw,uppderdir=x,lowerdir=y overlayfs /mount/point. Beyond that I'm clueless. I'm tinkering with it in a live system, but I haven't managed to get it to to work yet. I wish I could find out exactly what "upperdir" and "lowerdir" mean. I've found nothing. – Chuck R Mar 2 '12 at 16:17
up vote 59 down vote accepted

Edit: Since writing this answer, some things have changed in overlayfs, namely the addition of a required parameter workdir, see totti's answer below for a detailed description of this new parameter.

I finally managed to find it. I found references to it in the kernel source, but for some reason it doesn't appear in the git tree on kernel.org. But! If you pull the Ubuntu kernel source like this: apt-get source linux-image-3.0.0-16-generic you can find it in linux-3.0.0/Documentation/overlayfs.txt. It is also available in the linux-doc package in /usr/share/doc/linux-doc/filesystems/overlayfs.txt.gz.

As the actual help documentation is more of a "how it works" instead of a "how to mount with it," here's a brief rundown (there is one example in the kernel documentation):

mount -t overlayfs -o [mount options] overlayfs [mountpoint for merged system]

Where [mount options] can be:

  • lowerdir=somedir: lowerdir is the directory you're going to lay your new filesystem over, if there are duplicates these get overwritten by (actually, hidden in favor of) upperdir's version
  • upperdir=somedir: upperdir is the directory you want to overlay lowerdir with. If duplicate filenames exist in lowerdir and upperdir, upperdir's version takes precedence.
  • standard mount options. The only one I've seen from code is ro/rw, but you can experiment.

One thing that confused me at first, so I should probably clarify, is that mounting an overlayfs does not actually mount a filesystem. I was trying to mount a squashfs filesystem using an overlayfs mount, but that's not how it works. You must first mount the (in my case squashfs) filesystem to an arbitrary directory, then use overlayfs to merge the mount point (a directory) and another directory onto a tertiary directory (the overlayfs mount point)(edit: this "tertiary" directory can actually be the upperdir= directory). The tertiary directory is where you will see the merged filesystems (or directory trees - it's flexible).

Example 1, overlaying the root filesystem

I've been working on an Ubuntu hybrid boot disk where the base Ubuntu system exists as filesystem.squashfs and I have files called ubuntu.overlay kubuntu.overlay xubuntu.overlay and lubuntu.overlay. The .overlay files are base installs of said systems with the contents of filesystem.squashfs pruned (to save space). Then I modified the init scripts to overlay the correct distro's .overlay file (from a boot parameter) using overlayfs and the above options and it works like a charm!

These are the lines that I used in my init scripts (once all variables are translated):

mkdir -p /overlay
mount -t squashfs /cdrom/casper/ubuntu.overlay /overlay
mount -t overlayfs -o lowerdir=/filesystem.squashfs,upperdir=/overlay overlayfs /

Note that filesystem.squashfs above is a directory created by casper, not a file.

These three statements create an /overlay directory, mount a squashfs filesystem on the /overlay directory and then use OverlayFS to essentially merge the contents of /overlay over /.

Example 2, transparent merging of two directories

In the process of re-building my live USB for each release, I use OverlayFS to save a bunch of time. I start off with a directory called ubuntu-base which contains the contents of the ubuntu-core image which is the most basic install. I will then create directories called ubuntu, kubuntu, lubuntu, and xubuntu.

Then, I use OverlayFS to make the files from the ubuntu-base show up in the individual directories. I would use something like this:

mount -t overlayfs -o lowerdir=ubuntu-base,upperdir=kubuntu overlayfs kubuntu

This makes the files from ubuntu-base show up in the kubuntu folder. Then, I can chroot to the kubuntu folder and do something like apt-get install kubuntu-desktop. Any changes that are made while in this OverlayFS mount will remain in the upper directory, in this case the kubuntu folder. Then, once I unmount the OverlayFS mount the files that really exist in ubuntu-base but are "mirrored" into the kubuntu folder vanish unless they have been changed. This keeps me from having to have multiple copies of the files in ubuntu-base while still being able to use them as if they physically exist in each location.

  • 3
    "but for some reason it doesn't appear in the git tree on kernel.org" - That's because overlayfs is not in upstream kernel, just like aufs isn't (and won't ever be). Such union filesystems are integrated by Ubuntu Kernel Team. – MestreLion May 19 '12 at 13:18
  • 2
    OverlayFS is apparently getting into kernel 3.10. – David C. Bishop Mar 17 '13 at 3:57
  • 8
    Finally, should go in 3.18 lwn.net/Articles/617099 – Rmano Oct 20 '14 at 13:12
  • 4
    @Rmano: on my machine it works only with overlay not with overlayfs – Janus Troelsen Apr 6 '15 at 14:33
  • 1
    Thank you @austinmarton, I found this a while back too and totti covered it below before I updated my answer, so I gave credit to his answer at the top of mine. – Chuck R Apr 8 '16 at 6:11

From https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt:

Upper and Lower

An overlay filesystem combines two filesystems - an 'upper' filesystem and a 'lower' filesystem. When a name exists in both filesystems, the object in the 'upper' filesystem is visible while the object in the 'lower' filesystem is either hidden or, in the case of directories, merged with the 'upper' object.

It would be more correct to refer to an upper and lower 'directory tree' rather than 'filesystem' as it is quite possible for both directory trees to be in the same filesystem and there is no requirement that the root of a filesystem be given for either upper or lower.

The lower filesystem can be any filesystem supported by Linux and does not need to be writable. The lower filesystem can even be another overlayfs. The upper filesystem will normally be writable and if it is it must support the creation of trusted.* extended attributes, and must provide valid d_type in readdir responses, so NFS is not suitable.

A read-only overlay of two read-only filesystems may use any filesystem type.

Directories

Overlaying mainly involves directories. If a given name appears in both upper and lower filesystems and refers to a non-directory in either, then the lower object is hidden - the name refers only to the upper object.

Where both upper and lower objects are directories, a merged directory is formed.

At mount time, the two directories given as mount options "lowerdir" and "upperdir" are combined into a merged directory:

mount -t overlay overlay -olowerdir=/lower,upperdir=/upper,workdir=/work /merged

The "workdir" needs to be an empty directory on the same filesystem as upperdir.

Then whenever a lookup is requested in such a merged directory, the lookup is performed in each actual directory and the combined result is cached in the dentry belonging to the overlay filesystem. If both actual lookups find directories, both are stored and a merged directory is created, otherwise only one is stored: the upper if it exists, else the lower.

Only the lists of names from directories are merged. Other content such as metadata and extended attributes are reported for the upper directory only. These attributes of the lower directory are hidden.

  • 4
    Thanks for the clarification on workdir. That was a fairly recent change and for whatever reason I was never able to find any documentation about what 'workdir' actually did. It wasn't in the kernel help either last time I looked. Adds a layer of confusion to overlayfs IMO, wish the newer versions could be used without it -- it kind of messes up my workflow. – Chuck R Oct 27 '15 at 13:58

I have extended these artikels to include a Script for overlayfs that sets up a read-only root fs.

Hope it helps.

  • 1
    Thanks to the work that others did, Dustin and I have added a banary package named 'overlayroot' to quantal, which is part of the cloud-initramfs-tools source package. See usage documentation in /etc/overlayroot.conf – smoser Jul 31 '12 at 14:37

Minimal runnable example

# Create the filesystems.
dd if=/dev/zero of=lower.ext4 bs=1024 count=102400
mkfs -t ext4 lower.ext4
cp lower.ext4 upper.ext4
mkdir lower upper overlay
sudo mount lower.ext4 lower
sudo mount upper.ext4 upper
sudo chown "$USER:$USER" lower upper
printf lower-content > lower/lower-file
# Upper and work must be on the same filesystem.
mkdir upper/upper upper/work
printf upper-content > upper/upper/upper-file
# Work must be empty. E.g. this would be bad:
#printf work-content > upper/work/work-file
# Make the lower readonly to show that that is possible:
# writes actually end up on the upper filesystem.
sudo mount -o remount,ro lower.ext4 lower

# Create the overlay mount.
sudo mount \
  -t overlay \
  -o lowerdir=lower,upperdir=upper/upper,workdir=upper/work \
  none \
  overlay \
;

# Interact with the mount.
printf 'overlay-content' > overlay/overlay-file
ls lower upper/upper upper/work overlay

# Write to underlying directories while mounted
# gives undefined behaviour.
#printf lower-content-2 > lower/lower-file-2
#printf upper-content-2 > upper/upper-file-2

# Unmount the overlay and observe state.
sudo umount overlay
ls lower upper/upper upper/work

# Cleanup.
sudo umount upper lower

GitHub upstream.

Output of the first ls with the mount:

lower:
lost+found  lower-file

overlay:
lost+found  lower-file  overlay-file  upper-file

upper/upper:
overlay-file  upper-file

upper/work:
work

Output of the second ls without the mount:

lower:
lost+found  lower-file

upper/upper:
overlay-file  upper-file

upper/work:
work

Interpretation:

  • lower: was unchanged after writing to overlay
  • upper: received the modification to overlay
  • overlay: shows files from both upper and lower
  • work: contains some random content (a work/ directory) we should not care about

Example adapted from: Example OverlayFS Usage

Here is a more complex example with multiple lower layers: Overlayfs reload with multiple layers (migration away from aufs)

Tested on Ubuntu 18.04, Linux kernel 4.15.0.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.