How can I switch a folder to read-only mode so I can backup, but letting the server to write transparently to a journal file

Question

I have a very large server (Minecraft) using tens of gigabytes of files. Every two days, I make a backup, but since the server is so big, I have to shut it down for 45 minutes until the backup is done.

If I backup while the server is running, the backup folder gets corrupted because not all files were backed up at the same time and there's duplicate data, missing data and other problems. Chmodding the server folder recursively to 440 (read-only) makes the server spam thousands of errors, and sometimes crashes it or corrupts data.

What I am trying to do is to switch the server to read-only mode, but to allow it to write to temporary "journal" files which will be written to the real files once the backup is finished.

Is this possible at all? If yes, what tool should I use to do this? I was thinking something such switching the server folder to a soft link to a folder or partition that simulates this.

Answer 1

Using overlayfs

Use overlayfs on the root of your server (server folder).

1. Make a read-only link to the server folder, from which you can back up the data while the real directory is covered

   mkdir -p /path/to/ReadOnlyServerRoot sudo mount --bind /path/to/serverRoot /path/to/ReadOnlyServerRoot -o remount,ro

2. Shutdown the server so that files and databases are properly closed

3. Protect actual server root, /path/to/serverRoot, from write.

   sudo mount -t overlayfs overlayfs /path/to/serverRoot -o rw,uppderdir=/path/to/changes,lowerdir=/path/to/serverRoot
   

The changed and newly added data is stored at /path/to/changes. When a file is modified a new copy is made on /path/to/changes/path/to/file and this file is updated.

4. Start the server

5. Now you can backup your data from /path/to/ReadOnlyServerRoot

6. Shutdown server after the backup is completed

7. Remove the cover

   sudo umount /path/to/serverRoot
   

8. Use a synchronization software, such as rsync, to merge the change data into the server file.

   rsync /path/to/changes /path/to/serverRoot
   rm -rf /path/to/changes
   

9. Start the server

More information on on overlayfs's option can be found on this answer.

Steps 2, 4, 6, and 9 are optional, but highly recommended: At the time of creating and removing overlay on server folder it is highly recommended to shutdown your server (only seconds of downtime) for a successful backup as server may cache partial data on their memory and only a part of the data would be updated. As a result backup contains invalid/unusable data which in turn lead to lost data.

Using a file system snapshot

switch to a file system that support snapshot like btrfs or lvm

Update

If overlayfs is not available use aufs or unionfs
unionfs

    mount -t unionfs -o dirs=/branch_rw=rw:/branch_ro=rounionfs/union

creates a Union in directory /union with the branch directories /branch_rw(writable) and /branch_ro (read-only).

aufs

mkdir /tmp/dir1
mkdir /tmp/aufs-root
mount -t aufs -o br=/tmp/dir1:/tmp/dir2 none /tmp/aufs-root/

The first two commands created 2 new directories. The mount.aufs is the command to mount the filesystem as Union mount. The mount command, specifies it is going to union mount /tmp/dir1 and /tmp/dir2 under /tmp/aufs-root. The directory /tmp/aufs-root will have the content of both /tmp/dir1 and /tmp/dir2.

Answer 2

If you created your filesystem with Logical Volume Manager (LVM) enabled, then you could make a filesystem snapshot and then backup that snapshot while your server is never taken down.

Three-page tutorial here: http://www.howtoforge.com/linux_lvm_snapshots

Although it could mean that you will have to recreate your filesystem.