10

I was trying to install Ubuntu live on a USB-drive with persistent changes. However i wanted to write the changes to HDD instead of USB for various reasons (Performance, Space).

I found out, that the changes are handled within a file with a ext3 file-system that is mounted by the system (casper-rw). But it seem to be auto-detected on flash-drive on startup.

How can i create a new casper-rw file on HDD and reroute the Ubuntu live system on the USB-drive to HDD?

edit 2015-07-14:

Thank you for the answers so far. They are very interesting and help a lot to understand the mechanics of the "live system persistent save feature" better, and show good approaches to solve the problem. Unfortunately i was trying to overcome some limitations that i had in mind when asking the question regarding access to the host system and/or access to new hardware i did not clearly state (sorry for that).

If partitioning is an option, i think the answer stated here is very good, since it only requires you to create a new partition, label it casper-rw and you're done (work's great, i tested it).

If buying new hardware is an option, usb3.0 Flash-Drive (MLC or SLC), portable HDDs, eSATA and other options come to mind that have a far better performance than an average USB2.0 flash-drive.

So i was wondering if there is a way to just create a casper-rw file instead of a partition on a disk with limited access to (or copy/link the one on the flash-drive).

This way it would be easy to just copy the casper-rw file back to the flash-drive and use it from there again or move it to an other computer, granting great flexibility, minimal changes to the HDD and easy removal of changes done to the system.

  • Not sure if this is an option, but if you only need to save data, you could also just mount a HDD partition somewhere to your live file system tree... That way you only have specific folders located on the disk though. – Byte Commander Jul 1 '15 at 10:56
  • @ByteCommander thank you for your comment. Unfortunately partitioning the HDD is not an option for me and i would want everything to be as compact as possible so i can delete the data fast and simple, so a single file on HDD would be ideal. – L.S. Jul 1 '15 at 11:50
  • It should be possible to mount a file containing an image of a file system (e.g. .iso or .img, probably more) too. Unfortunately I never did this and can't tell you how to do so... – Byte Commander Jul 1 '15 at 17:42
  • How do you know this is not working ? Is your file named casper-rw ? Is it at the root of your partition ? I have done that (using persistence on HDD with a partition labelled casper-rw instead) and I don't recall any extra necessary step. – solsTiCe Jul 10 '15 at 9:02
  • Your latest edit makes your question too specific to be of broad use (which my answer is...) It's not impossible, but would mean creating scripts and udev rules to [unmount]-[synchronise to HDD]-[mount] on insertion and [ synchronise to HDD] on unmount, but that's a ton of work and not really suited for a Q&A site like this one. – Fabby Jul 15 '15 at 21:51
4
+100

You can easily put the writable filesystem (in the casper-rw file) for a live media onto a hard disk. The limitation is that the casper-rw file must go on a FAT partition. Newer machines (UEFI) all have a FAT EFI partition, but that's typically too small to hold a 1G-4G casper-rw file. On another big enough FAT partition, you can make directories, each holding a casper-rw file for possibly different live medias. Suppose sda11 is 10G and has a 10G FAT filesystem, mounted at /mnt/sda11,on which there are directories /A , /B , /C , /D , and /E. Assume we will use /A for our persistent media, putting a casper-rw there.

cd /mnt/sda11/A
dd if=/dev/zero of=casper-rw bs=1M  count=4096
mkfs.ext4 -F -O^has_journal -L casper-rw casper-rw

Take your live media created with persistence, and edit the /boot/grub/grub.cfg file and the /syslinux/txt.cfg file, adding after the word "persistent"

"persistent-path=/A"

/boot/grub/grub.cfg ...

menuentry "Try Ubuntu without installing" {
    set gfxpayload=keep
    linux   /casper/vmlinuz.efi  file=/cdrom/preseed/ubuntu.seed boot=casper quiet splash --- cdrom-detect/try-usb=true noprompt persistent persistent-path=/A
    initrd  /casper/initrd.lz
}

/syslinux/txt.cfg

default live
label live
menu label ^Try Ubuntu without installing
kernel /casper/vmlinuz.efi
append noprompt cdrom-detect/try-usb=true persistent persistent-path=/A file=/cdrom/preseed/ubuntu.seed boot=casper initrd=/casper/initrd.lz quiet splash ---
label live-install
...

That's it. You don't even need to rename/remove the casper-rw file on the USB media.


If there's room on the USB media, you can even copy the hard disk's casper-rw back to the USB, and take your changes with you.


The persistent-path does not allow any explicit disk reference, so should be unique across all FAT partitions. Tested with 1 or 2 FAT partitions (one being the EFI partition). Will not work on an ext2 or ntfs filesystem instead of FAT. If you also add the "toram" word on the same line as "persistent", your compressed filesystem on the slow USB will be copied into ram and give much better performance, however, there seems to be a shutdown issue, with the FAT partition not being cleanly unmounted (which does not seem to cause any problems but...)

  • Thanks. That makes things much clearer. I assume that in the boot state ntfs can't be recognized (right?) thus making it impossible to have a casper-rw on a ntfs partition. So i think i'll go either with the ext4 partition labeled casper-rw or try some "toram" or ramdrive stuff. – L.S. Jul 16 '15 at 20:32
  • How do you modify grub.cfg or syslinux.txt given that you could only mount the live media read only as it is an iso ?? – solsTiCe Jan 4 '16 at 19:35
  • The above used the ISO to create a live-media USB, on which the files are writeable. Booting the ISO directly (off a hard disk for instance) uses an entirely separate grub (the one on the hard disk) whose files are editable -- this case not using the boot mechanisms from the ISO at all. – ubfan1 Jan 5 '16 at 4:46
7

That's extremely easy if you use the following set-up:

  1. Install Ubuntu on an SLC USB stick (USB 3.0 preferred if your hardware supports it) without any of the casper stuff (just a normal install, treating the USB stick as an SSD.)

    Why? The SLCs are twice as expensive as the MLCs but they are 4 times faster and last 8* longer! So they really are small SSDs on a stick!

  2. Optimize the system and treat the USB stick as a small SSD, so:

    In your /etc/sysctl.conf add:

    # Fabby: change the "swappiness" to 10 to prevent swapping as much as possible
    # to not wear out the USB stick as the Ubuntu default is optimized for a server.
    # 10 to balance with vfs_cache_pressure
    vm.swappiness = 10
    
    # Fabby: Lower vfs_cache_pressure to 75% 
    # (once cached, probably not immediately needed any more)
    #
    # This percentage value controls the tendency of the kernel to reclaim
    # the memory which is used for caching of directory and inode objects.
    #
    # At the default value of vfs_cache_pressure=100 the kernel will attempt to
    # reclaim dentries and inodes at a "fair" rate with respect to pagecache and
    # swapcache reclaim.  Decreasing vfs_cache_pressure causes the kernel to prefer
    # to retain dentry and inode caches.
    vm.vfs_cache_pressure = 75
    
    # Fabby: Good to improve sequential reads (stop stuttering in movie play)
    # Can also be implemented per disk using udev rules
    vm.max-readahead=2048
    vm.min-readahead=1024
    

    In your /etc/fstab add the bold paramaters to your boot USB line. (It'll probably be sdc or sdd. The below is an example, use UID if possible)

    /dev/sdc / ext4 defaults,noatime,discard,errors=remount-ro 0 1

  3. create 2 users: an admin user which has its home directory on the USB stick but has very little files and a normal user that will have its home directory on the HDD.

  4. Only now insert the HDD and add the necessary HDD info to your fstab to auto-mount the HDD and add the following parameters:

    nobootwait,nofail

    to allow booting even when the HDD is not present (If that's the case, only the admin user will have access, not the normal user)

  5. Now log in as the admin user and move the home directory for the normal user from the USB stick to the HDD:

cp --preserve=all /home/NormalUser/* /media/ExternalHDD/NormalUser

If all is OK:

rm --recursive --force /home/NormalUser/*
cd /home
ln --symbolic /media/ExternalHDD/NormalUser

Done! :-)

For the record: I have a 16GB USB 2.0 MLC stick just like this (more then enough for my use-case and not too expensive and it's about 5 years old now and still running!)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.