I am using Ubuntu 18.04 OS and I installed some components for example ROS, Cuda, folly, TensorRT etc. Now if I will change computer or my computer will crash, I will setup all components manually. How can I copy this OS with all this components to another computer? Or for my computer, If I run into a problem in the future, how can I setup my OS with all components rapidly?
Here's one approach I use: Split the system into three parts: Base OS, Data, Key Applications
- Base OS is the stock Ubuntu system.
- Data is separate from the applications that process data.
- Key Applications (that process data sets) are containerized.
Backups: Only data is frequently backed up (it's valuable). The Key Applications container is snapshotted regularly for easy reversion if a problem occurs.
Upgrades: The OS and the Container are separate; an upgrade of one won't break the other. If an application upgrade breaks the stack in a container, I can revert to the last snapshot.
Migration: I can easily copy the container(s) and data to a different system without re-installing anything.
I can also clone the container for testing risky alternative applications or different versions. Or I can spin up a fresh container to rebuild everything from scratch. All without risking my working container, or my data, or the stability of my base OS.
There is some learning and maintenance involved: I use cron, LXD, and duplicity for snapshots, containers, and backups. I keep notes on how to install/uninstall everything in my containers, and links to those projects' instructions. I track the release schedules of the Base OS and Key Applications so I'm not surprised by a change. I set aside time to test changes and updates.
I will add to @BulletBob's recommendation for Clonezilla. This is the simplest and probably fastest way to recover from a crashed system. Here is an overview of the process:
- Download Clonezilla and create a bootable USB. The steps are provided on the web site.
- Get an external hard drive. It need not be an SSD, but should have enough space for the image of your computer's disk. Personally, I backup every month and keep 3 versions of backups. The backups might be much smaller than your primary disk, so you won't need exactly 3x of your primary disk.
- Shut down your computer. Keep the external drive ready but DO NOT ATTACH IT NOW. Plug in the CLonezilla USB that you created above.
- Start the computer and boot off the USB. You might need to hit some key like F12 to get to the boot menu and ask the computer to boot off the USB.
- Accept the defaults. Choose the Disk to image option (not partitions). You can skip the disk checks -- your call.
- Choose the source disk (this is your primary disk) that you want to make an image of.
- At some point (depending on your version of Clonezilla), it will ask you to insert the destination disk. This is when you attach the external drive.
- Select the external drive destination location.
- Pick a location and backup image name or accept the defaults.
- Kick off the backup. It might take a while. If you have a 250G drive, it can take up to 1 hour, depending on the speed of the USB which your external drive is connected to.
- When done, follow the instructions to poweroff.
- Remove the Clonezilla bootable USB and boot the system as usual. The external drive is still attached.
- Once the system is up and running, verify that the backup files are present in the external drive.
- Detach the drive and store it.
To recover, the process is simple.
- Install a new disk (assuming the old one crashed).
- Attach the external disk.
- Boot off the Clonezilla USB.
- Choose the restore option and go through the steps which are self-explanatory. Make sure to get the source and destination disks right.
I would recommend RescueZilla https://rescuezilla.com/
It is use Clonezilla under the hood, but so much easier to use. Do a Backup and a Restore in just a few click. But like clonezilla, the system must be booted from the USB or DVD.
Personally I use ReaR, that I can schedule via a cron job,while system is running. I have done a couple of restore so far and it work well. https://relax-and-recover.org/
Image-based backups have all sorts of flaws, but mainly the inconvenience of down time, huge amounts of storage, and the inflexibility that other methods provide are the real problems.
Look to file-based backup methods for greater efficiency, more flexibility, and repeatability - not just to the exact same system, but for OS clean upgrades or moving to new systems with very different hardware.
With image-based backups, just 2 copies requires 2x the storage. Whereas with file-based backups, 90+ days of daily, versioned, backups, probably will need 10-30% more storage than the source - say 100G that would be 200G with images or 130G for 90+days of versioned backups. There are many excellent file-based backup methods that support versioning. With versions, you can see what changed from day to day and perhaps figure out what is breaking the system? Tools like rsnapshot, rdiff-backup, back-in-time are just what is needed.
The detailed steps are too long to post here. Sorry. The short answer is:
- Don't backup the OS
- DO backup OS settings, personal settings, server settings
- DO backup personal files
- DO backup server files like RDBMS files (probably want to dump those)
- DO include a list of manually installed APT, Snap, Flakpak packages in your backups.
- DO keep manually compiled, non-packaged, stuff in /usr/local/ and back that area up.
- DO use cron to automate backups after testing. Manual backups never seem to actually be run. Automate at least weekly, if not daily, backup jobs.
- DO get/look at post-backup reports to see if they worked for the first few weeks. Can't say how many times I've been called to a client to help restore only to find their backups have been failing for over a year, but nobody was checking or watching.
- DO test the restore process. Except 5-10 test runs will be needed to get it right. It is likely at least 1 chicken/egg problem will arise. Until a good restore happens, you don't actually know.