I would like to add a different recommendation to the pool of possible solutions. I would recommend you to base your setup on btrfs' subvolume and snapshot abilities in combination with a btrbk cronjob.
Setting this up is not necessarily trivial but largely depends on your skill set and previous experience. There is a lot of literature on the net that will help you. In the end you are rewarded with a very flexible and fast way to backup your SSD regularly with a lot of options to create a solution perfect for your needs.
Note of caution: Any form of raid can and should never replace regular backups. (Luckily btrbk can easily be extended to external drives or ssh reachable host, see its manual)
General setup
The general ideas of my proposal is to use the SSD as your btrfs system drive that contains your root and related subvolumes and the two HDDs in a btrfs raid1 as your data and backup drive.
Together with btrbk this will allow you to perform automated incremental backups of your system SSD to your backup HDDs. And since the HDDs are set up as a mirror, all your backup will be kept mirrored as well.
Furthermore btrfs ability to send and receive subvolumes (which is what btrbk utilizes to make backups) allow you to freely move your data and backups between your system and data drive. This will allow you to change what data is stored on the fast SSD, while always maintaining versioned and mirrored backups of all your data.
Setup btrfs
To get started you will either need to reinstall Ubuntu onto the SSD and selecting btrfs as your root file system or to convert your existing installations file system to btrfs. Both ways are described on the Ubuntu's community help page about btrfs, which is a good read in general, if you are just getting start with btrfs on Ubuntu.
Next you need to turn the HDDs into a btrfs raid1, with the following command, where /dev/sdx
and /dev/sdy
are the two drives (All data on these drives will be lost!):
mkfs.btrfs -d raid1 /dev/sdx /dev/sdy
If you are new to snapshots or btrfs, I would recommend the this moment to familiarize yourself with the difference between folders, subvolumes and snapshots and try out some of the commands, before any actual data is written to your raid1.
There are many ways you can organize your data and you can find some examples in the btrfs kernel wiki's sysadmin guide.
One way to do it is to mount your btrfs root (subvolid
0
or 5
) somewhere and use that to manage your subvolumes and snapshot and furthermore store all of the data in appropriate subvolumes, which you mount to convenient locations in your file system. That way you can snapshot, move, recover and replace any data at will.
For your concrete example, that could mean the following (all command should be run as root/with sudo):
- Mount your system btrfs root (
subvolid=0
) to /btrfs/system
- Mount your data btrfs raid1 root (
subvolid=0
) to /btrfs/data
Instead of mounting these volumes by hand, add them to your fstab (/etc/fstab
) before mounting, so they are mounted at boot as well. I would recommend to mount them by their UUID, which you can retrieve by running sudo btrfs filesystem show
.
UUID=<UUID of system> /btrfs/system btrfs defaults,subvolid=0 0 0
UUID=<UUID of data> /btrfs/data btrfs defaults,subvolid=0 0 0
Now mount them with:
sudo mkdir /btrfs
sudo mkdir /btrfs/data
sudo mount /btrfs/data
sudo mkdir /btrfs/system
sudo mount /btrfs/system
Now you can any additional subvolumes you might want to each of the btrfs filesystems.
Ubuntu normally creates a subvolume for your root /
(subvol=@
) and home directory /home
(subvol=@home
) by default. It is common to turn /var
or /tmp
into their own subvolumes or to create application specific subvolumes, e.g. for /var/www/
.
Personally I prefer to keep all my subvolumes at the btrfs root and them mount them to their specific locations using mount and fstab entries.
For example, to create a subvolume for your music collection on the HDD raid1, I would do the following:
btrfs subvolume create /btrfs/data/@music
I would then mount it with the following fstab entry to /music
:
UUID=<UUID of data> /music btrfs defaults,subvol=@music 0 0
Setup of btrbk
Secondly, you will have to set up btrbk for the subvolumes you want to snapshot and backup onto the HDD raid.
As a simple example on how to backup @
and @home
regularly and keeping regular spaced history of your backup you could write the following to /etc/btrbk.conf
:
# The long timestamp recommended for more then one snapshot a day
timestamp_format long
# Set time spacing of snapshots kept on SSD
snapshot_preserve_min 2d
snapshot_preserve 7d 4w 3m
# Set time spacing of snapshots kept on HDD raid
target_preserve_min no
target_preserve 8w *m
snapshot_dir /btrfs/system/snapshots
volume /btrfs/system
subvolume @
target send-receive /btrfs/data/backup/
subvolume @home
target send-receive /btrfs/data/backup/
Please read the btrbk documentation for any of the details. It will also explain how to recover your data from a snapshot.
Lastly you will have to add btrbk to your crontab with sudo crontab -e
. E.g. to run your btrbk snapshots and backup every day at noon, add the following line:
0 12 * * * /path/of/btrbk run
Other considerations
Swap
While in general there is less and less need for swap space in modern personal computer systems that have at least 8 GB of RAM, there are still use cases where it can help you out, especially when located on a SSD, where the performance hit of swapping is not as noticeable. It is therefore still generally recommended to set up a swap file or partition.
That being said, btrfs does not support swap files. That means, you will have to allocate some of your SSD space into a separate swap partition if you want to be able to use swap on your system at all.
btrfs SSD detection
Btrfs automatically detects if a file system mounted is located on a SSD and enables wear-leveling in that case.
This however is not necessary anymore, as modern SSDs automatically wear-level themselves, while at the same time causing issues with the fragmentation of free space. I would therefore personally advice you to mount your SSD with the nossd
option.
More details can be found in the btrfs kernel wiki.
File system compression
Btrfs supports transparent file compression. By adding the compress
option to your mount flags, it will be enable for all new files written.
For example, to enable compression for the @music
subvolume I used as an example earlier, I would change my fstab entry to:
UUID=<UUID of data> /music btrfs defaults,compress,subvol=@music 0 0
To apply this change, do not forget to remount (unmount and mount again) the effected subvolume.
With most recent computers the overhead of compressing files before it is being writing and after it has been read is often negligible. Writing a large but well compressible file to a slow disk might even be faster with compression enabled.
If you are worried about speed, you can also use the faster but less efficient LZO compression with compress=lzo
.
More details once again can be found in the btrfs kernel wiki.
Location of home folder
Ubuntu puts your home folder into the @home
subvolume on the system drive by default. If you move home to your HDD mirror is up to your personal preferences. You can also keep it on the SSD and include it into your btrbk backup as I showed above.
Time spacing of backups
While my example set the same retention times for all subvolumes, btrbk allows you to set this time for each of them individually.
You could also run btrbk with different configs (see -c
option) at different intervals to gain even more control over when what subvolume is snapshotted amd/or backuped.
As snapshots are quickly created, thanks to btrfs copy-on-write nature, you could even get btrbk to create a snapshot each hour, but only transfer them to the backup disk once a day, for folders that are changed often, like your home folder.
Detecting bit rot
To make use of btrfs bit rot/data corruption detection (and automatic repair in case of your raid1), you should make sure to run a btrfs scrub at regular intervals, e.g. as a cron job, with:
btrfs scrub start /btrfs/system
btrfs scrub start /btrfs/data