How do I use it? Where is its documentation?
I finally managed to find it. I found references to it in the kernel source, but for some reason it doesn't appear in the git tree on kernel.org. But! If you pull the Ubuntu kernel source like this:
As the actual help documentation is more of a "how it works" instead of a "how to mount with it," here's a brief rundown (there is one example in the kernel documentation):
Where [mount options] can be:
One thing that confused me at first, so I should probably clarify, is that mounting an overlayfs does not actually mount a filesystem. I was trying to mount a squashfs filesystem using an overlayfs mount, but that's not how it works. You must first mount the (in my case squashfs) filesystem to an arbitrary directory, then use overlayfs to merge the mount point (a directory) and another directory onto a tertiary directory (the overlayfs mount point)(edit: this "tertiary" directory can actually be the upperdir= directory). The tertiary directory is where you will see the merged filesystems (or directory trees - it's flexible).
Example 1, overlaying the root filesystem
I've been working on an Ubuntu hybrid boot disk where the base Ubuntu system exists as filesystem.squashfs and I have files called ubuntu.overlay kubuntu.overlay xubuntu.overlay and lubuntu.overlay. The .overlay files are base installs of said systems with the contents of filesystem.squashfs pruned (to save space). Then I modified the init scripts to overlay the correct distro's .overlay file (from a boot parameter) using overlayfs and the above options and it works like a charm!
These are the lines that I used in my init scripts (once all variables are translated):
Note that filesystem.squashfs above is a directory created by casper, not a file.
These three statements create an /overlay directory, mount a squashfs filesystem on the /overlay directory and then use OverlayFS to essentially merge the contents of /overlay over /.
Example 2, transparent merging of two directories
In the process of re-building my live USB for each release, I use OverlayFS to save a bunch of time. I start off with a directory called ubuntu-base which contains the contents of the ubuntu-core image which is the most basic install. I will then create directories called ubuntu, kubuntu, lubuntu, and xubuntu.
Then, I use OverlayFS to make the files from the ubuntu-base show up in the individual directories. I would use something like this:
This makes the files from ubuntu-base show up in the kubuntu folder. Then, I can
Upper and Lower
An overlay filesystem combines two filesystems - an 'upper' filesystem and a 'lower' filesystem. When a name exists in both filesystems, the object in the 'upper' filesystem is visible while the object in the 'lower' filesystem is either hidden or, in the case of directories, merged with the 'upper' object.
It would be more correct to refer to an upper and lower 'directory tree' rather than 'filesystem' as it is quite possible for both directory trees to be in the same filesystem and there is no requirement that the root of a filesystem be given for either upper or lower.
The lower filesystem can be any filesystem supported by Linux and does not need to be writable. The lower filesystem can even be another overlayfs. The upper filesystem will normally be writable and if it is it must support the creation of trusted.* extended attributes, and must provide valid d_type in readdir responses, so NFS is not suitable.
A read-only overlay of two read-only filesystems may use any filesystem type.
Overlaying mainly involves directories. If a given name appears in both upper and lower filesystems and refers to a non-directory in either, then the lower object is hidden - the name refers only to the upper object.
Where both upper and lower objects are directories, a merged directory is formed.
At mount time, the two directories given as mount options "lowerdir" and "upperdir" are combined into a merged directory:
The "workdir" needs to be an empty directory on the same filesystem as upperdir.
Then whenever a lookup is requested in such a merged directory, the lookup is performed in each actual directory and the combined result is cached in the dentry belonging to the overlay filesystem. If both actual lookups find directories, both are stored and a merged directory is created, otherwise only one is stored: the upper if it exists, else the lower.
Only the lists of names from directories are merged. Other content such as metadata and extended attributes are reported for the upper directory only. These attributes of the lower directory are hidden.
I have extended these artikels to include a Script for overlayfs that sets up a read-only root fs.
Hope it helps.