7

Persistent live drive with a partition for persistence

There are some web sites describing

  • new tools
  • new features in [new versions of] old tools

that can create a persistent live drive from an iso file with Ubuntu 19.10 and the corresponding Ubuntu family flavours.

  • Why and how does it work?
  • How is it different from earlier methods for persistent live drives?
  • Isn't live the default for a cloned drive?
8

Persistent live drive with a partition for persistence

Yes, there are some web sites describing new tools and new features in [new versions of] old tools that can create a persistent live drive from an iso file with Ubuntu 19.10 and the corresponding Ubuntu family flavours.

  • A new feature or fixed bug ('feature request' or 'bug fix' according to this link) makes it possible to use a partition for persistence, when booting directly from a cloned or extracted copy from the iso file.

    This makes it much easier to create a live drive with a partition for persistence with Ubuntu 19.10.

  • In previous versions of Ubuntu it was possible to use a file in a FAT32 file system for persistence. Its size is limited to 4 GiB. It was possible to boot via grub in a separate partition (from the content of the iso file) and use a partition for persistence. But this is more complicated so not suitable for manual methods.

  • In 19.10 and future versions the size of a partition for persistence is only limited by the size of the drive (USB pendrive, SSD, HDD, memory card).

  • Debian 10 live iso files have also this feature, and can be used to create persistent live drives with a partition for persistence.

Tools

Ubuntu and Debian

  • mkusb - works with all current versions of Ubuntu

    If you run standard Ubuntu live, you need an extra instruction to get the repository Universe. (Kubuntu, Lubuntu ... Xubuntu have the repository Universe activated automatically.)

    sudo add-apt-repository universe  # only for standard Ubuntu
    
    sudo add-apt-repository ppa:mkusb/ppa  # and press Enter
    sudo apt-get update
    sudo apt-get install mkusb mkusb-nox usb-pack-efi
    

    enter image description here

  • mkusb-minp - small stand-alone shellscript that can use this new feature in Ubuntu 19.10

    Please read the warnings when you try older versions or re-spins (distros based on Ubuntu)

    Beginning of dialogue: enter image description here

    End of dialogue: enter image description here

    The whole dialogue (as text without high-lighting):

    lubuntu@lubuntu:~/Downloads$ ls
    lubuntu-19.10-desktop-amd64.iso  mkusb-minp
    
    lubuntu@lubuntu:~/Downloads$ ./mkusb-minp 
     Run 'mkusb-minp' with sudo or as root 
                   This shellsript wraps a safety belt around 'dd'               
    Clone from an iso or image file to a target device (typically a USB pendrive)
    You may want to install 'pv' (progress view)
    Usage:
     sudo /path/mkusb-minp [-p] <source file> <target device> 
    Cloning:
     sudo ./mkusb-minp file.iso /dev/sdx
     sudo ./mkusb-minp file.img /dev/sdx
    Persistence: for Ubuntu 19.10+ and Debian 10+
     sudo ./mkusb-minp -p file.iso /dev/sdx
    Help:
    ./mkusb-minp -h
    Version:
    ./mkusb-minp -v
    Available devices (tran-MODEL  device),
     mmc-SP32G_0x52460eb1  /dev/mmcblk0 
     ata-LITEONIT_LMT-128M6M_mSATA_128GB_TW0T8MRJ550854170015  /dev/sda 
     usb-SanDisk_Extreme_AA010806122254222903-0:0  /dev/sdb 
    
    lubuntu@lubuntu:~/Downloads$ sudo ./mkusb-minp -p lubuntu-19.10-desktop-amd64.iso /dev/mmcblk0
                   This shellsript wraps a safety belt around 'dd'               
    It helps you select the target device in a safe way to avoid
    overwriting valuable data by mistake.  Please double-check 
    at the final checkpoint! [press Enter to continue] 
    
     Please check that you will clone to the correct target device! 
     mmc-SP32G_0x52460eb1  /dev/mmcblk0 
     ata-LITEONIT_LMT-128M6M_mSATA_128GB_TW0T8MRJ550854170015  /dev/sda 
     usb-SanDisk_Extreme_AA010806122254222903-0:0  /dev/sdb 
    NAME        MODEL FSTYPE  LABEL               MOUNTPOINT                          SIZE NAME
    mmcblk0                                                                          29,7G mmcblk0
    ├─mmcblk0p1       ntfs    usbdata             /media/lubuntu/usbdata             13,9G mmcblk0p1
    ├─mmcblk0p2                                                                         1M mmcblk0p2
    ├─mmcblk0p3       vfat    usbboot                                                 244M mmcblk0p3
    ├─mmcblk0p4       iso9660 Lubuntu 19.10 amd64 /media/lubuntu/Lubuntu 19.10 amd64  1,7G mmcblk0p4
    └─mmcblk0p5       ext4    casper-rw           /media/lubuntu/casper-rw1          13,9G mmcblk0p5
    
     Not a USB device. Do you really want to overwrite  /dev/mmcblk0  ? (y/N) y      
     Final checkpoint                                                                                                               
    Do you want to overwrite  /dev/mmcblk0 ? (y/N) y                                                                                
    mount: /tmp/tmp.alUfbjCcgA: WARNING: device write-protected, mounted read-only.                                                 
    #define DISKNAME  Lubuntu 19.10 "Eoan Ermine" - Release amd64                                   
     Ubuntu or an Ubuntu family flavour 
     Making persistent live drive of version 19.10 
    --------------------------------------------------------------------------------
              ***** puer: Persistence for Ubuntu *****
    Trying to unmount partitions if mounted on the target device
    umount: /dev/mmcblk0: not mounted.
    umount: /dev/mmcblk0p2: not mounted.
    umount: /dev/mmcblk0p3: not mounted.
    --------------------------------------------------------------------------------
     Please wait until the process has finished and 'Done' is written 
    ..... Flash modified iso file to target ........................................
    ..... Create partition for persistence .........................................
    
    Welcome to fdisk (util-linux 2.34).                                                              
    Changes will remain in memory only, until you decide to write them.                              
    Be careful before using the write command.
    
    
    Command (m for help): Partition type
       p   primary (2 primary, 0 extended, 2 free)
       e   extended (container for logical partitions)
    Select (default p): Partition number (3,4, default 3): First sector (3283712-62333951, default 3284992): Last sector, +/-sectors or +/-size{K,M,G,T,P} (3284992-62333951, default 62333951): 
    Created a new partition 3 of type 'Linux' and of size 28,2 GiB.
    
    Command (m for help): The partition table has been altered.
    Calling ioctl() to re-read partition table.
    Syncing disks.
    
    ..... Overwrite first mibibyte of partition for persistence ....................
    1024+0 records in
    1024+0 records out
    1048576 bytes (1,0 MB, 1,0 MiB) copied, 0,190755 s, 5,5 MB/s
    ..... Create file system in partition for persistence ..........................
    mke2fs 1.45.3 (14-Jul-2019)
    Discarding device blocks: done                            
    Creating filesystem with 7381120 4k blocks and 1847776 inodes
    Filesystem UUID: 6a73fc77-af36-4d55-abe0-6963f7e60b1f
    Superblock backups stored on blocks: 
            32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
            4096000
    
    Allocating group tables: done                            
    Writing inode tables: done                            
    Writing superblocks and filesystem accounting information: done   
    
    Press Enter to continue, when you have unplugged the target device '/dev/mmcblk0'
    and maybe have plugged it back again
    --------------------------------------------------------------------------------
    NAME        MODEL FSTYPE  LABEL               MOUNTPOINT  SIZE NAME
    mmcblk0           iso9660 Lubuntu 19.10 amd64            29,7G mmcblk0
    ├─mmcblk0p1       iso9660 Lubuntu 19.10 amd64             1,6G mmcblk0p1
    ├─mmcblk0p2       vfat    Lubuntu 19.10 amd64             3,9M mmcblk0p2
    └─mmcblk0p3       ext2    casper-rw                      28,2G mmcblk0p3
     Done :-) 
    lubuntu@lubuntu:~/Downloads$ 
    

Windows

  • New: Simple method where the iso file is edited before cloning

    This new feature in Ubuntu 19.10 can be used also from Windows,

    • Use HxD (a binary editor) to edit the iso file in order to replace two cosmetic boot options 'quiet splash' with 'persistent ' (replace 12 characters with 12 characters)
    • Use Win32 Disk Imager to clone the edited iso file
    • The first time the cloned drive is booted, the Ubuntu system will create a casper-rw partition with an ext4 file system automatically. Simple and robust!

    • There are detailed descriptions at these links:

      enter image description here

      enter image description here

  • Rufus - a well-known tool that can use this new feature in Ubuntu 19.10. Be sure to use the current version (Rufus 3.8 or newer),

    • works well to create live-only drives with all current Ubuntu versions,
    • can create persistent live drives with Ubuntu 19.10 and future versions, but the experimental feature in Rufus, that creates an ext3 file system in the casper-rw partition is buggy. The developer is working on a solution.
    • Please read the changelog at the web site, and the warnings (pop-up windows), they can help you avoid serious mistakes.

      enter image description here

MacOS and Windows and Linux

  • Unetbootin - you have to create a casper-rw partition for persistence manually and remove or rename the casper-rw file

    See this link describing how to add a persistent partition to a UNetbootin live/persistent USB (Ubuntu 19.10+)

    Please notice 'Space used to preserve files across reboots ...'

    enter image description here

Manual method

You need only a few manual steps to create a persistent live drive with a partition for persistence, when you are running Ubuntu.

  • Edit the iso file to replace quiet splash with persistent. Yes, you can edit the binary iso file and replace 12 characters with 12 other characters and flash the output to the target device (usually a USB pendrive). sed can do it.

  • Create a partition 'behind' the flashed copy of the edited iso file. fdisk can do it.

  • Create an ext2 file system in this partition and put the label casper-rw on this partition. mkfs.ext2 can do it.

  • Flush the buffers. sync can do it.

This is what mkusb-minp is doing plus a lot of checking, that wrap a safety belt around the process. In other words, you get help to write to the correct target device and get warnings when there might be problems.

You can do it yourself and feel every step, or you can read the code of the shellscript mkusb-minp and understand the details.

Please notice that I use dus and select 'Cloning' in the following example. You can use another cloning tool, but if you use a simple tool like dd, please double-check that you are cloning to the currect device, and that all partitions on that device are unmounted.

strings lubuntu-19.10-desktop-amd64.iso |grep 'quiet splash'  # check that 'quiet splash' is there to be replaced by 'persistent  ' (12 characters)
sed 's/quiet splash/persistent  /' lubuntu-19.10-desktop-amd64.iso > persistent-lubuntu-19.10-desktop-amd64.iso  # yes, sed works with binary files
ls -l *19.10*  # check that the size is the same
strings persistent-lubuntu-19.10-desktop-amd64.iso |grep 'persistent  '  # check that 'persistent' is there now
dus persistent-lubuntu-19.10-desktop-amd64.iso  # I use mkusb-dus, you can use the Ubuntu Startup Disk Creator or another cloning tool
sudo lsblk -fm  # It is important to check the device letter of the target drive (the USB drive, that you want persistent live
sudo fdisk /dev/sdx  # x is the device letter of the target drive, please double-check that you have the correct letter
n           # new partition
p           # primary
<Return>    # default: 3
<Return>    # default: next free sector
<Return>    # default: last addressable sector
w           # write and quit
sudo lsblk -fm  # check that things look good and verify that partition #3 is the correct partition to be used to store the persistent data
sudo mkfs.ext2 -L casper-rw /dev/sdx3  # put label and file system into the partition of persistence
sudo lsblk -fm  # check that things look good
sync  # flush the buffers and wait for prompt

Links

There are general manual methods that can be easily modified from using a file for persistence to a partition for persistence. See the following link,

A cloned Ubuntu 19.10+ live drive is not really live-only

Isn't live the default for a cloned drive?

Well, it used to be, and it is still live but not live-only in 19.10.

If you want to make a purely live-only drive, where nothing will be preserved after shutdown and reboot, you can

  • use mkusb-minp with the option -n

  • or manually modify the built-in boot options to replace 'quiet splash' with 'nopersistent'

Link

  • 2
    This looks like an authoritative answer to a canonical question that will get a large number of page views. For this reason I think adding screenshots would be appropriate. – karel Oct 18 at 12:14
  • 1
    @karel, Good idea :-) Please advice which screenshots to add (and how many to avoid drowning in screenshots). – sudodus Oct 18 at 12:16
  • How about illustrating the persistence feature, similar to the screenshots in this answer? If the question looks like canonical material this will protect it from being reflex closed by reviewers. – karel Oct 18 at 12:18
  • @karel, OK. It should be possible to link to that very picture, and to add a picture for mkusb and maybe another one or two. – sudodus Oct 18 at 12:20
  • 1
    @karel, Done :-) Please check and give feedback - if you think it is OK or for mkusb-minp maybe I should reduce the screenshots and/or remove the code list with the whole dialogue. – sudodus Oct 18 at 17:27
0

Rufus is the easiest tool for making a persistent live drive with Ubuntu 19.10

(But not the necessarily the best).

For making a Persistent drive not limited to 4GB in Ubuntu, mkusb is easiest, (and best), to use. (What else is there)?

For making a Persistent drive not limited to 4GB in Windows 10, Rufus seems to currently be the easiest tool to use.

That does not make it the best tool though.

I did some simple experiments comparing the latest version of Rufus and mkusb:

First I wiped a 4GB USB and then ran Rufus in Windows 10, rebooted, and had a look ar the /casper-rw/upper/ folder with Disk Usage Analyzer.

Then I deleted the casper-rw partition and made a new one.

I rebooted the drive a couple times, without making any changes or saving anything, and again had a look at the repopulated Persistent partition with DUA.

I again wiped the drive and created another Persistent USB with mkusb, rebooted and had another look with DUA.

Again I deleted the casper-rw partition and made a new one.

I rebooted the drive a couple times, without making any changes, and again had a look at the repopulated Persistent partition with DUA.

As can be seen from the attached charts, mkusb seems to be superior in the use of disk space, and grub 2 is more straight forward than syslinux.

Persistent USB made with Rufus

Persistent USB made with Rufus

Persistent USB made with Rufus repopulated

Persistent USB made with Rufus with Repopulated casper-rw partition

Persistent USB made with mkusb

Persistent USB made with mkusb

Persistent USB made with mkusb repopulated

Persistent USB made with mkusb with Repopulated casper-rw partition

Edit: following the trail of persistent disk space usage.

I took a look at disk space usage in the casper-rw partition of Rufus and mkusb freshly made persistent USB's

Rufus var folder

Rufus var folder

contents of Rufus snaps folder

Contents of Rufus snaps folder

![mkusb var folder

mkusb var folder

![contents of mkusb snaps folder

Contents of mkusb snaps folder

Data not yet confirmed

  • 1
    Rufus developer here. You do realize that free space is directly correlated to how many inodes are set as reserved during the formatting process, which is a value that is more or less arbitrary. At the moment, Rufus sets 5% aside for reserved blocks (ext2fs_r_blocks_count_set()), so I would surmise that, if mkusb went for less, then of course it will look like mkusb makes better use of space, even as what might happen in the long run is that you're going to be short of inodes to create new files as the drive fills... For the record, Rufus uses the default values from mke2fs.conf. – Akeo Oct 23 at 13:04
  • 2
    Also, you'll have to explain your "grub 2 is more straight forward than syslinux" comment, since it would seem that you are under the impression that Rufus forces Syslinux, which is an erroneous assumption. The use of Syslinux vs GRUB is decided by BIOS vs UEFI boot, and is exactly as it has been set by the Ubuntu maintainers. Rufus does not force any over the other but instead completely respects the maintainer's decision over which one to use depending of the boot mode. – Akeo Oct 23 at 13:07
  • @Akeo: In my humble opinion you are doing great work with Rufus, By great work I mean that you are helping save Windows users from a life of misery by helping introduce them to Linux. However I have seen comments in these pages concerning use of space in Rufus Persistent partitions. I am trying to do a little leg work to try to improve the situation .(Not everyone here in Sri Lanka can afford humongous flash drives). Concerning grub/syslinux, perhaps my preference for Grub2 is a personal preference. I have had problems in the past with syslinux, UEFI and multiple partitions.. Best regards – C.S.Cameron Oct 23 at 15:21
  • 1
    @Akeo. Please read the results listed at this link. I am sorry to say it, but the problem concerning use of space in Rufus Persistent partitions increases dramatically with increasing size of the partition. To make the point clear: In a 60 GB SSD connected via a USB adapter, 40 GiB are occupied, and only 12 GiB are available for the user. After fixing with e2fsck -f the drive is almost as good as a fresh drive made by mkusb. So there is a real problem with the file system. Please take a second look at it. – sudodus Oct 23 at 15:32
  • 1
    @sudodus, this chain of comments is why this would better dealt with in the issue tracker. I do not alter release binaries after they are published, so, no, the binary you used is the same. It is also digitally signed, so there is no point in providing an MD5, because if it got altered in any way from what I uploaded on release day, the signature validation will fail and Windows will tell you. And yes, I am planning to look into this issue, but it's being made more complicated due to the fact that I am using official e2fsprogs code (the same as what Linux uses) which I didn't write myself. – Akeo Oct 24 at 18:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.