16

Persistent live drive with a partition for persistence

There are some web sites describing

  • new tools
  • new features in [new versions of] old tools

that can create a persistent live drive from an iso file with Ubuntu 19.10 and the corresponding Ubuntu family flavours.

  • Why and how does it work?
  • How is it different from earlier methods for persistent live drives?
  • Isn't live the default for a cloned drive?
16

Persistent live drive with a partition for persistence

Yes, there are some web sites describing new tools and new features in [new versions of] old tools that can create a persistent live drive from an iso file with Ubuntu 19.10 and the corresponding Ubuntu family flavours.

  • A new feature or fixed bug ('feature request' or 'bug fix' according to this link) makes it possible to use a partition for persistence, when booting directly from a cloned or extracted copy from the iso file.

    This makes it much easier to create a live drive with a partition for persistence with Ubuntu 19.10.

  • In previous versions of Ubuntu it was possible to use a file in a FAT32 file system for persistence. Its size is limited to 4 GiB. It was possible to boot via grub in a separate partition (from the content of the iso file) and use a partition for persistence. But this is more complicated so not suitable for manual methods.

  • In 19.10 and future versions the size of a partition for persistence is only limited by the size of the drive (USB pendrive, SSD, HDD, memory card).

  • Debian 10 live iso files have also this feature, and can be used to create persistent live drives with a partition for persistence.

  • Newest new: In the developing Focal Fossa, to be released as 20.04 LTS, the default label of the partition for persistence is changed from casper-rw to writable. This is implemented in mkusb-plug version 2.5.5, (which is bundled with mkusb version 12.4.3 in the PPA).

Tools

Ubuntu and Debian

  • mkusb - works with all current versions of Ubuntu

    If you run standard Ubuntu live, you need an extra instruction to get the repository Universe. (Kubuntu, Lubuntu ... Xubuntu have the repository Universe activated automatically.)

    sudo add-apt-repository universe  # only for standard Ubuntu
    
    sudo add-apt-repository ppa:mkusb/ppa  # and press Enter
    sudo apt-get update
    sudo apt-get install mkusb mkusb-nox usb-pack-efi
    

    enter image description here

  • mkusb-minp - small stand-alone shellscript that can use this new feature in Ubuntu 19.10. It works with Debian 10 too.

    Please read the warnings when you try older versions or re-spins (distros based on Ubuntu)

    Beginning of dialogue: enter image description here

    End of dialogue: enter image description here

  • mkusb-plug - small set of shellscripts that can use this new feature in Ubuntu 19.10. It works with Debian 10 too. I think you will find things easier with this new and very safe tool with a graphical user interface.

    enter image description here

    enter image description here

    enter image description here

Windows

  • Update: Rufus - a well-known tool that can use this new feature in Ubuntu 19.10. Be sure to use the current version (Rufus 3.9 or newer),

    • works well to create live-only drives with all current Ubuntu versions,
    • can create persistent live drives with Ubuntu 19.10 and future versions. Please upgrade to version 3.9 (or newer), which creates a good ext3 file system and can make good persistent live drives.
    • Please read the changelog at the web site, and the warnings (pop-up windows), they can help you avoid serious mistakes.

    • Edit: Rufus 3.9 has arrived :-)

      enter image description here

  • Simple method where the iso file is edited before cloning

    This new feature in Ubuntu 19.10 can be used also from Windows,

    • Use HxD (a binary editor) to edit the iso file in order to replace two cosmetic boot options 'quiet splash' with 'persistent ' (replace 12 characters with 12 characters)
    • Use Win32 Disk Imager to clone the edited iso file
    • The first time the cloned drive is booted, the Ubuntu system will create a casper-rw partition with an ext4 file system automatically. Simple and robust!

    • There are detailed descriptions at these links:

      enter image description here

      enter image description here

MacOS and Windows and Linux

  • Unetbootin - you have to create a casper-rw partition for persistence manually and remove or rename the casper-rw file

    See this link describing how to add a persistent partition to a UNetbootin live/persistent USB (Ubuntu 19.10+)

    Please notice 'Space used to preserve files across reboots ...'

    enter image description here

  • Compressed image file

    You can extract and clone a compressed image file directly according to the following link,

    Xubuntu Core 20.04 LTS with mkusb 12.4.5 (mkusb-dus and mkusb-plug)

    This works in all operating systems where you have

    • a tool that can extract a file compressed with xz
    • a cloning tool

    . enter image description here

Manual method

You need only a few manual steps to create a persistent live drive with a partition for persistence, when you are running Ubuntu.

  • Edit the iso file to replace quiet splash with persistent. Yes, you can edit the binary iso file and replace 12 characters with 12 other characters and flash the output to the target device (usually a USB pendrive). sed can do it.

  • Create a partition 'behind' the flashed copy of the edited iso file. fdisk can do it.

  • Create an ext2 file system in this partition and put the label casper-rw on this partition. mkfs.ext2 can do it.

  • Flush the buffers. sync can do it.

This is what mkusb-minp is doing plus a lot of checking, that wrap a safety belt around the process. In other words, you get help to write to the correct target device and get warnings when there might be problems.

You can do it yourself and feel every step, or you can read the code of the shellscript mkusb-minp and understand the details.

Please notice that I use dus and select 'Cloning' in the following example. You can use another cloning tool, but if you use a simple tool like dd, please double-check that you are cloning to the currect device, and that all partitions on that device are unmounted.

strings lubuntu-19.10-desktop-amd64.iso |grep 'quiet splash'  # check that 'quiet splash' is there to be replaced by 'persistent  ' (12 characters)
sed 's/quiet splash/persistent  /' lubuntu-19.10-desktop-amd64.iso > persistent-lubuntu-19.10-desktop-amd64.iso  # yes, sed works with binary files
ls -l *19.10*  # check that the size is the same
strings persistent-lubuntu-19.10-desktop-amd64.iso |grep 'persistent  '  # check that 'persistent' is there now
dus persistent-lubuntu-19.10-desktop-amd64.iso  # I use mkusb-dus, you can use the Ubuntu Startup Disk Creator or another cloning tool
sudo lsblk -fm  # It is important to check the device letter of the target drive (the USB drive, that you want persistent live
sudo fdisk /dev/sdx  # x is the device letter of the target drive, please double-check that you have the correct letter
n           # new partition
p           # primary
<Return>    # default: 3
<Return>    # default: next free sector
<Return>    # default: last addressable sector
w           # write and quit
sudo lsblk -fm  # check that things look good and verify that partition #3 is the correct partition to be used to store the persistent data
sudo mkfs.ext2 -L casper-rw /dev/sdx3  # put label and file system into the partition of persistence
sudo lsblk -fm  # check that things look good
sync  # flush the buffers and wait for prompt

Links

There are general manual methods that can be easily modified from using a file for persistence to a partition for persistence. See the following link,

A cloned Ubuntu 19.10+ live drive is not really live-only

Isn't live the default for a cloned drive?

Well, it used to be, and it is still live but not live-only in 19.10.

If you want to make a purely live-only drive, where nothing will be preserved after shutdown and reboot, you can

  • use mkusb-minp with the option -n or mkusb-plug and select 'No-persistent live drive'.

  • or manually modify the built-in boot options to replace 'quiet splash' with 'nopersistent'

Link

| improve this answer | |
  • 2
    This looks like an authoritative answer to a canonical question that will get a large number of page views. For this reason I think adding screenshots would be appropriate. – karel Oct 18 '19 at 12:14
  • 1
    @karel, Good idea :-) Please advice which screenshots to add (and how many to avoid drowning in screenshots). – sudodus Oct 18 '19 at 12:16
  • How about illustrating the persistence feature, similar to the screenshots in this answer? If the question looks like canonical material this will protect it from being reflex closed by reviewers. – karel Oct 18 '19 at 12:18
  • @karel, OK. It should be possible to link to that very picture, and to add a picture for mkusb and maybe another one or two. – sudodus Oct 18 '19 at 12:20
  • 1
    @karel, Done :-) Please check and give feedback - if you think it is OK or for mkusb-minp maybe I should reduce the screenshots and/or remove the code list with the whole dialogue. – sudodus Oct 18 '19 at 17:27
0

Rufus is the easiest tool for making a persistent live drive with Ubuntu 19.10

(But not the necessarily the best).

For making a Persistent drive not limited to 4GB in Ubuntu, mkusb is easiest, (and best), to use. (What else is there)?

For making a Persistent drive not limited to 4GB in Windows 10, Rufus seems to currently be the easiest tool to use.

That does not make it the best tool though.

I did some simple experiments comparing the latest version of Rufus and mkusb:

First I wiped a 4GB USB and then ran Rufus in Windows 10, rebooted, and had a look ar the /casper-rw/upper/ folder with Disk Usage Analyzer.

Then I deleted the casper-rw partition and made a new one.

I rebooted the drive a couple times, without making any changes or saving anything, and again had a look at the repopulated Persistent partition with DUA.

I again wiped the drive and created another Persistent USB with mkusb, rebooted and had another look with DUA.

Again I deleted the casper-rw partition and made a new one.

I rebooted the drive a couple times, without making any changes, and again had a look at the repopulated Persistent partition with DUA.

As can be seen from the attached charts, mkusb seems to be superior in the use of disk space, and grub 2 is more straight forward than syslinux.

Persistent USB made with Rufus

Persistent USB made with Rufus

Persistent USB made with Rufus repopulated

Persistent USB made with Rufus with Repopulated casper-rw partition

Persistent USB made with mkusb

Persistent USB made with mkusb

Persistent USB made with mkusb repopulated

Persistent USB made with mkusb with Repopulated casper-rw partition

Edit: following the trail of persistent disk space usage.

I took a look at disk space usage in the casper-rw partition of Rufus and mkusb freshly made persistent USB's

Rufus var folder

Rufus var folder

contents of Rufus snaps folder

Contents of Rufus snaps folder

![mkusb var folder

mkusb var folder

![contents of mkusb snaps folder

Contents of mkusb snaps folder

Data not yet confirmed

Edit: Rufus 3.9.1624

Ran Disk Usage Analyzer with the latest version of Rufus, Results were close to Rufus 3.8. It apears that mkusb still makes a more space efficient Persistent drive.

enter image description here Persistent USB made with Rufus 3.9 enter image description here Rufus 3.9 var folder

Again, Data and methodology has not been confirmed.

| improve this answer | |
  • 2
    Rufus developer here. You do realize that free space is directly correlated to how many inodes are set as reserved during the formatting process, which is a value that is more or less arbitrary. At the moment, Rufus sets 5% aside for reserved blocks (ext2fs_r_blocks_count_set()), so I would surmise that, if mkusb went for less, then of course it will look like mkusb makes better use of space, even as what might happen in the long run is that you're going to be short of inodes to create new files as the drive fills... For the record, Rufus uses the default values from mke2fs.conf. – Akeo Oct 23 '19 at 13:04
  • 2
    Also, you'll have to explain your "grub 2 is more straight forward than syslinux" comment, since it would seem that you are under the impression that Rufus forces Syslinux, which is an erroneous assumption. The use of Syslinux vs GRUB is decided by BIOS vs UEFI boot, and is exactly as it has been set by the Ubuntu maintainers. Rufus does not force any over the other but instead completely respects the maintainer's decision over which one to use depending of the boot mode. – Akeo Oct 23 '19 at 13:07
  • @Akeo: In my humble opinion you are doing great work with Rufus, By great work I mean that you are helping save Windows users from a life of misery by helping introduce them to Linux. However I have seen comments in these pages concerning use of space in Rufus Persistent partitions. I am trying to do a little leg work to try to improve the situation .(Not everyone here in Sri Lanka can afford humongous flash drives). Concerning grub/syslinux, perhaps my preference for Grub2 is a personal preference. I have had problems in the past with syslinux, UEFI and multiple partitions.. Best regards – C.S.Cameron Oct 23 '19 at 15:21
  • 1
    @Akeo. Please read the results listed at this link. I am sorry to say it, but the problem concerning use of space in Rufus Persistent partitions increases dramatically with increasing size of the partition. To make the point clear: In a 60 GB SSD connected via a USB adapter, 40 GiB are occupied, and only 12 GiB are available for the user. After fixing with e2fsck -f the drive is almost as good as a fresh drive made by mkusb. So there is a real problem with the file system. Please take a second look at it. – sudodus Oct 23 '19 at 15:32
  • 1
    @sudodus, this chain of comments is why this would better dealt with in the issue tracker. I do not alter release binaries after they are published, so, no, the binary you used is the same. It is also digitally signed, so there is no point in providing an MD5, because if it got altered in any way from what I uploaded on release day, the signature validation will fail and Windows will tell you. And yes, I am planning to look into this issue, but it's being made more complicated due to the fact that I am using official e2fsprogs code (the same as what Linux uses) which I didn't write myself. – Akeo Oct 24 '19 at 18:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.