Howto cleanly unmount a root filesystem that's using aufs

Question

Preface: I've made a boot-image that boots up Debian/Ubuntu using SquashFS and aufs. The image contains a cleaned up debootstrap install, compressed into a squashfs image.

The squashfs image is considered the readonly part of my filesystem. The local harddrive is considered the writable part of my root filesystem.

All setup of the rootfs is done from initramfs which calls switch_root to boot-up Debian/Ubuntu. The rootfs itself is mounted using aufs on the squash image (ro) and the harddrive (rw).

When the initramfs is running, I check if the harddrive is cleanly unmounted. I make sure it's clean before I mount it by calling 'fsck -Cp' (I know, I shouldn't).

The Problem: The problem that I have, is that the writable part of my root filesystem is never unmounted properly, leaving it (eg, the hdd) in an unclean state.

The base of the problem is obviously that the hdd is in use by the aufs filesystem. The aufs filesystem is obviously in use by the rootfs. Next to that, I figure that the Debian/Ubuntu shutdown procedure must be modified to allow this.

What have I done so far: I have tried to shutdown by hand, by first going into runlevel 1, and forcing ro-remounts of the rootfs and the hdd, but the hdd is still never unmounted cleanly.

I also changed the way aufs is mounted by adding an option to create the xino file outside the harddrive mount. I never see the file being created, nor can I see any open files on the harddrive using lsof.

Last but not least, I added the harddrive to the fstab of the Ubuntu install to have it recognize it. Didn't change a thing when Debian/Ubuntu shuts down though :(

From the looks of it, I have created a "chicken and egg" problem, as I can't unmount the harddrive without first unmounting the root filesystem, which I need to unmount the harddrive lol.

I haven't got the faintest idea of what commands to run, in order to unmount it properly, so any help is very much appreciated.

Answer 1

You can never unmount root filesystem, sorry -- no matter if you're using squashfs+aufs, or just plain ext2/ext3/ext4...

What you can do is remount it read-only, which is the same from data integrity standpoint. For regular filesystem:

mount -oremount,ro /
sync # initiate writing any pending writes to disk
sleep 5 # and give writes some time to finish.

note that aufs is somewhat different as you should make sure the rw-backing of it still writeable while aufs itself is being put read-only... for example, assuming /mnt/sq/rw is ext4 read-write part of your aufs "/" mount:

 cat /proc/mounts > /etc/mtab # may not be needed if your mtab is symlink or otherwise reflects real situation
 fuser -mv / # show debug of what is using this disk and blocking unmount/ro-remount, if anything
 auplink / flush 
 mount -oremount,noxino,noplink,ro /
 mount -oremount,sync /mnt/sq/rw
 mount -oremount,mod:/mnt/sq/rw=ro /
 mount -oremount,ro /mny/sq/rw
 sync
 sleep 5
 cat /proc/mounts
 dmesg
 sleep 300 # to give you time to read mounts/dmesg info
 halt -d -f

in production you obviously won't need debug info...

In short, you flush to disk and disable aufs features that require writing to ext4 overlay, then force ext4 itself to sync and go read-only, so it will be marked as cleanly unmounted next time you boot.