How to create a live system on usb-drive with persistent changes on disk/hdd

Question

I was trying to install Ubuntu live on a USB-drive with persistent changes. However i wanted to write the changes to HDD instead of USB for various reasons (Performance, Space).

I found out, that the changes are handled within a file with a ext3 file-system that is mounted by the system (casper-rw). But it seem to be auto-detected on flash-drive on startup.

How can i create a new casper-rw file on HDD and reroute the Ubuntu live system on the USB-drive to HDD?

edit 2015-07-14:

Thank you for the answers so far. They are very interesting and help a lot to understand the mechanics of the "live system persistent save feature" better, and show good approaches to solve the problem. Unfortunately i was trying to overcome some limitations that i had in mind when asking the question regarding access to the host system and/or access to new hardware i did not clearly state (sorry for that).

If partitioning is an option, i think the answer stated here is very good, since it only requires you to create a new partition, label it casper-rw and you're done (work's great, i tested it).

If buying new hardware is an option, usb3.0 Flash-Drive (MLC or SLC), portable HDDs, eSATA and other options come to mind that have a far better performance than an average USB2.0 flash-drive.

So i was wondering if there is a way to just create a casper-rw file instead of a partition on a disk with limited access to (or copy/link the one on the flash-drive).

This way it would be easy to just copy the casper-rw file back to the flash-drive and use it from there again or move it to an other computer, granting great flexibility, minimal changes to the HDD and easy removal of changes done to the system.

Answer 1

That's extremely easy if you use the following set-up:

1. Install Ubuntu on an SLC USB stick (USB 3.0 preferred if your hardware supports it) without any of the casper stuff (just a normal install, treating the USB stick as an SSD.)

   Why? The SLCs are twice as expensive as the MLCs but they are 4 times faster and last 8* longer! So they really are small SSDs on a stick!

2. Optimize the system and treat the USB stick as a small SSD, so:

   In your /etc/sysctl.conf add:

   # Fabby: change the "swappiness" to 10 to prevent swapping as much as possible
   # to not wear out the USB stick as the Ubuntu default is optimized for a server.
   # 10 to balance with vfs_cache_pressure
   vm.swappiness = 10
   
   # Fabby: Lower vfs_cache_pressure to 75% 
   # (once cached, probably not immediately needed any more)
   #
   # This percentage value controls the tendency of the kernel to reclaim
   # the memory which is used for caching of directory and inode objects.
   #
   # At the default value of vfs_cache_pressure=100 the kernel will attempt to
   # reclaim dentries and inodes at a "fair" rate with respect to pagecache and
   # swapcache reclaim.  Decreasing vfs_cache_pressure causes the kernel to prefer
   # to retain dentry and inode caches.
   vm.vfs_cache_pressure = 75
   
   # Fabby: Good to improve sequential reads (stop stuttering in movie play)
   # Can also be implemented per disk using udev rules
   vm.max-readahead=2048
   vm.min-readahead=1024
   

   In your /etc/fstab add the bold paramaters to your boot USB line. (It'll probably be sdc or sdd. The below is an example, use UID if possible)

   /dev/sdc / ext4 defaults,noatime,discard,errors=remount-ro 0 1

3. create 2 users: an admin user which has its home directory on the USB stick but has very little files and a normal user that will have its home directory on the HDD.

4. Only now insert the HDD and add the necessary HDD info to your fstab to auto-mount the HDD and add the following parameters:

   nobootwait,nofail

   to allow booting even when the HDD is not present (If that's the case, only the admin user will have access, not the normal user)

5. Now log in as the admin user and move the home directory for the normal user from the USB stick to the HDD:

cp --preserve=all /home/NormalUser/* /media/ExternalHDD/NormalUser

If all is OK:

rm --recursive --force /home/NormalUser/*
cd /home
ln --symbolic /media/ExternalHDD/NormalUser

Done! :-)

For the record: I have a 16GB USB 2.0 MLC stick just like this (more then enough for my use-case and not too expensive and it's about 5 years old now and still running!)

Answer 2

You can easily put the writable filesystem (in the casper-rw file) for a live media onto a hard disk. The limitation is that the casper-rw file must go on a FAT partition. Newer machines (UEFI) all have a FAT EFI partition, but that's typically too small to hold a 1G-4G casper-rw file. On another big enough FAT partition, you can make directories, each holding a casper-rw file for possibly different live medias. Suppose sda11 is 10G and has a 10G FAT filesystem, mounted at /mnt/sda11,on which there are directories /A , /B , /C , /D , and /E. Assume we will use /A for our persistent media, putting a casper-rw there.

cd /mnt/sda11/A
dd if=/dev/zero of=casper-rw bs=1M  count=4096
mkfs.ext4 -F -O^has_journal -L casper-rw casper-rw

Take your live media created with persistence, and edit the /boot/grub/grub.cfg file and the /syslinux/txt.cfg file, adding after the word "persistent"

"persistent-path=/A"

/boot/grub/grub.cfg ...

menuentry "Try Ubuntu without installing" {
    set gfxpayload=keep
    linux   /casper/vmlinuz.efi  file=/cdrom/preseed/ubuntu.seed boot=casper quiet splash --- cdrom-detect/try-usb=true noprompt persistent persistent-path=/A
    initrd  /casper/initrd.lz
}

/syslinux/txt.cfg

default live
label live
menu label ^Try Ubuntu without installing
kernel /casper/vmlinuz.efi
append noprompt cdrom-detect/try-usb=true persistent persistent-path=/A file=/cdrom/preseed/ubuntu.seed boot=casper initrd=/casper/initrd.lz quiet splash ---
label live-install
...

That's it. You don't even need to rename/remove the casper-rw file on the USB media.

---

If there's room on the USB media, you can even copy the hard disk's casper-rw back to the USB, and take your changes with you.

---

The persistent-path does not allow any explicit disk reference, so should be unique across all FAT partitions. Tested with 1 or 2 FAT partitions (one being the EFI partition). Will not work on an ext2 or ntfs filesystem instead of FAT. If you also add the "toram" word on the same line as "persistent", your compressed filesystem on the slow USB will be copied into ram and give much better performance, however, there seems to be a shutdown issue, with the FAT partition not being cleanly unmounted (which does not seem to cause any problems but...)