How do I make a persistent live USB of Ubuntu 16.04?

Question

I need the persistent live Ubuntu of preferably 16.04 because I need to rescue my whole encrypted HDD of 16.04 as discussed in the thread How do I add myself back in sudo group in Encrypted HDD of 16.04? I create here a new persistent Live Ubuntu 16.04 with some bugs described at the end with Videonauth.

1. sudo apt-get install gksu debootstrap
2. gksudo gparted. Make USB with ext4 leaving only 1GB as free memory, and set 1GB for Linux Swap; Manage flags > only boot.

3. Prepare the OS by and run in the USB stick

   sudo mkdir /mnt/stick
   sudo mount /dev/sdb1 /mnt/stick/
   sudo debootstrap --arch=amd64 xenial /mnt/stick http://de.archive.ubuntu.com/ubuntu/
   sudo mount -o bind /dev /mnt/stick/dev
   sudo mount -o bind /dev/pts /mnt/stick/dev/pts
   sudo mount -t sysfs /sys /mnt/stick/sys
   sudo mount -t proc /proc /mnt/stick/proc
   sudo cp /proc/mounts /mnt/stick/etc/mtab
   sudo cp /etc/resolv.conf /mnt/stick/etc/resolv.conf
   sudo chroot /mnt/stick/
   

4. Prompt looks like root@masi-CM6340:/#. Set locales.

   %locale-gen en_US en_US.UTF-8
   %dpkg-reconfigure locale
   dpkg-rekonfigure locales % two above commands cause a bug in gnome-terminal; choose here your locales and set environment en_US.UTF-8
   dpkg-reconfigure keyboard-configuration
   localedef -i en_US -c -f UTF-8 en_US.UTF-8
   

5. apt-get install linux-image-generic. You get /dev/sda [NotThisOne, your HDD], 2 /dev/sdb (31474 MB; ???) 2b - /dev/sdb1 (30398 MB; /). Choose /dev/sdb1 and /dev/sdb. Choese here Linux kernel what you like. I keep Linux kernel 4.6 most stable which you can download as described at the end of the body.

6. apt-get install vim wget

7. blkid, copy to clipboard, vim /etc/fstab. Remove sda lines so only two sdb lines. Edit them to like UUID="..." swap swap defaulst 0 0 and UUID="..." / ext4 defaults 0 1.

8. Paste the following to the file /etc/apt/sources.list

   #deb cdrom:[Ubuntu 16.04 LTS _Xenial Xerus_ - Release amd64 (20160420.1)]/ xenial main restricted
   
   # See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
   # newer versions of the distribution.
   deb http://de.archive.ubuntu.com/ubuntu/ xenial main restricted
   # deb-src http://de.archive.ubuntu.com/ubuntu/ xenial main restricted
   
   ## Major bug fix updates produced after the final release of the
   ## distribution.
   deb http://de.archive.ubuntu.com/ubuntu/ xenial-updates main restricted
   # deb-src http://de.archive.ubuntu.com/ubuntu/ xenial-updates main restricted
   
   ## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
   ## team, and may not be under a free licence. Please satisfy yourself as to
   ## your rights to use the software. Also, please note that software in
   ## universe WILL NOT receive any review or updates from the Ubuntu security
   ## team.
   deb http://de.archive.ubuntu.com/ubuntu/ xenial universe
   # deb-src http://de.archive.ubuntu.com/ubuntu/ xenial universe
   deb http://de.archive.ubuntu.com/ubuntu/ xenial-updates universe
   # deb-src http://de.archive.ubuntu.com/ubuntu/ xenial-updates universe
   
   ## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu 
   ## team, and may not be under a free licence. Please satisfy yourself as to 
   ## your rights to use the software. Also, please note that software in 
   ## multiverse WILL NOT receive any review or updates from the Ubuntu
   ## security team.
   deb http://de.archive.ubuntu.com/ubuntu/ xenial multiverse
   # deb-src http://de.archive.ubuntu.com/ubuntu/ xenial multiverse
   deb http://de.archive.ubuntu.com/ubuntu/ xenial-updates multiverse
   # deb-src http://de.archive.ubuntu.com/ubuntu/ xenial-updates multiverse
   
   ## N.B. software from this repository may not have been tested as
   ## extensively as that contained in the main release, although it includes
   ## newer versions of some applications which may provide useful features.
   ## Also, please note that software in backports WILL NOT receive any review
   ## or updates from the Ubuntu security team.
   deb http://de.archive.ubuntu.com/ubuntu/ xenial-backports main restricted universe multiverse
   # deb-src http://de.archive.ubuntu.com/ubuntu/ xenial-backports main restricted universe multiverse
   
   ## Uncomment the following two lines to add software from Canonical's
   ## 'partner' repository.
   ## This software is not part of Ubuntu, but is offered by Canonical and the
   ## respective vendors as a service to Ubuntu users.
   deb http://archive.canonical.com/ubuntu xenial partner
   deb-src http://archive.canonical.com/ubuntu xenial partner
   
   deb http://security.ubuntu.com/ubuntu xenial-security main restricted
   # deb-src http://security.ubuntu.com/ubuntu xenial-security main restricted
   deb http://security.ubuntu.com/ubuntu xenial-security universe
   # deb-src http://security.ubuntu.com/ubuntu xenial-security universe
   deb http://security.ubuntu.com/ubuntu xenial-security multiverse
   # deb-src http://security.ubuntu.com/ubuntu xenial-security multiverse
   

9. Do

   dpkg-divert --local --rename --add /sbin/initctl
   ln -s /bin/true /sbin/initctl
   apt-get update
   % Comment this out if ubuntu-desktop is wanted
   apt-get install ubuntu-desktop 
   % Comment out if ubuntu-server is wanted
   % apt-get install ubuntu-server
   adduser --home /home/masi masi
   

10. Add masi to the existing groups usermod -aG adm,cdrom,sudo,dip,plugdev,lpadmin,sambashare masi where you can leave out sambashare.

11. Ignore drivers because USB stick not made for one PC.

12. For recovering the encrypted HDD, install the packages apt-get ecryptfs-utils schroot testdisk where testdisk for preparing discs and the other two for recovering the whole disc encrypted Ubuntu 16.04.

13. sudo echo 'GRUB_DISABLE_OS_PROBER="true"' >> /etc/default/grub; update-grub

14. Finalize the install. Do

    rm /sbin/initctl
    dpkg-divert --local --rename --remove /sbin/initctl
    

Test your Installation of Persistent Live Ubuntu 16.04 [alpha]

1. Start your new Persistent Ubuntu. In terminal, do sudo true. If you get Unable to resolve host * Connection refused, do

   # https://askubuntu.com/q/59458/25388
   sudo vim /etc/hostname
   masi 
   
   sudo vim /etc/hosts 
   127.0.0.1    localhost.localdomain masi
   [...]
   

2. Possible complication: Ubuntu starts in $(inittramfs)$. Do ...

Result: USB starts up Ubuntu Desktop GUI and UI

• you can open LibreOffice, Firefox, ...
• masi is in sudo group; groups correct.
• Locales set correctly when chroot to stick.
• If you have a bug in gnome-terminal i.e. opening it in GUI, do sudo dpkg-rekonfigure locales again in TTY1. It resolves the case.

Bugs which are reported to Videonauth

Always update-and-upgrade system in doing the tests: sudo apt-get update && sudo apt-get upgrade.

1. Ubuntu's Internet Browser. Launching it crashes the system. Mouse only works but no signal can be send to the TTY.
2. Firefox. Launching it gives a warning about [profile exists] so cannot open. Temporary fix rm -r ~/.mozilla && rm -r ./.cache/mozilla only for one session. The problem persists on every login. Better working fix: sudo mv -v /home/masi /home/masi_backup, sudo mkdir -v /home/masi, sudo chown masi:masi /home/masi, sudo chmod 755 /home/masi and sudo reboot because some ownerships of home-folder causes the bug. Run the command of the source and you get the following differences
3. /etc/hostname fails to match /etc/hosts after some time
4. Sometimes, starts up to (inittramsfs)

Output of vimdiff <(find /home/masi -printf "%P %u:%g %m\n" | sort) <(find /home/masi_backup -printf "%P %u:%g %m\n" | sort)

%LHS-file
.bash_history masi:masi 600
.cache/compizconfig-1/animation.pb masi:masi 664
.cache/compizconfig-1/commands.pb masi:masi 664
.cache/compizconfig-1/compiztoolbox.pb masi:masi 664
% here many files not existing in the RHS

%RHS-file (fresh installation)
.bash_history root:root 600
.bash_logout masi:masi 664
.bashrc masi:masi 644
.cache root:root 700

where I do not understand which caused the bug.

Upgrade to Linux Kernel 4.6

Do as instructed here. There is an universal bug with resume/suspend in Linux kernel 4.4. Upgrading to 4.6 solves it and makes the system much better.

System: 14.04, 16.04
Linux kernel: 4.4 - 4.6

Answer 1

To make yourself a complete normal Ubuntu installation on an USB or an external HDD you can follow the procedure I describe here.

You want to start by doing some preparatory installations, you need gksudo which is in the package gksu, schroot and debootstrap. Those packages are not installed by default, so type in a terminal (ctrl+alt+t):

sudo apt-get install gksu debootstrap schroot

Now you can start preparing your USB/HDD for installation by:

gksudo gparted

First you want to go to the devices menu and select create partition table, when asked for the type choose msdos. Now you should see an entry for unallocated space. Click on this and choose new and make it ext4 leaving only 1GB as free memory, then again click on the unallocated space entry and make it 1GB for Linux Swap. After this you apply your settings to the USB/HDD. Then right-click on the ext4 entry and choose manage flags and there you tick the check-mark on boot and leave.

Now you need to prepare your chroot environment to work in by doing the following commands one by one:

sudo mkdir /mnt/stick
sudo mount /dev/sdb1 /mnt/stick/
sudo debootstrap --arch=amd64 xenial /mnt/stick http://archive.ubuntu.com/ubuntu/
sudo mount -o bind /dev /mnt/stick/dev
sudo mount -o bind /dev/pts /mnt/stick/dev/pts
sudo mount -t sysfs /sys /mnt/stick/sys
sudo mount -t proc /proc /mnt/stick/proc
sudo cp /proc/mounts /mnt/stick/etc/mtab
sudo cp /etc/resolv.conf /mnt/stick/etc/resolv.conf
sudo chroot /mnt/stick/

Your prompt should now look like this:

root@HOSTNAME:/#

Now you can begin your real installation process, start by setting the locale setting and the keyboard-configuration and installing the kernel.

locale-gen en_US.UTF-8
dpkg-reconfigure locale
dpkg-reconfigure keyboard-configuration
localedef -i en_US -c -f UTF-8 en_US.UTF-8

apt-get install linux-image-generic

While this installation GRUB asks you for which device to be installed for. you will see a selection like this:

[ ]/dev/sda [NotThisOne, this is your HDD]
[ ]/dev/sdb (31474 MB; ???)
[ ]/dev/sdb1 (30398 MB; /)

Choose /dev/sdb1 and /dev/sdb by moving to the respective field using the arrow keys and pressing space then finally hit return. Now after this installation is completed you can install your editor of choice, I do here as example nano and vim:

apt-get install nano vim

Now you can edit your /etc/fstab file, for this first get a list of the UUIDS in your system by:

blkid

This should give you an output like this:

/dev/sda1: UUID="8f8f130f-e127-43b6-b797-8bfa3f1e1631" TYPE="ext4" PARTUUID="06bd4abe-01"
/dev/sda5: UUID="8469c58b-b2bb-4551-b47d-c1a7d21e9ebd" TYPE="swap" PARTUUID="06bd4abe-05"
/dev/sdb1: UUID="b08ab271-5619-479c-aa21-8aea7f4e6f3b" TYPE="ext4" PARTUUID="5872a1f8-01"
/dev/sdb2: UUID="c3309cee-731e-4030-93b9-f1d88c12c806" TYPE="swap" PARTUUID="5872a1f8-02"

For easier editing you can copy the output from terminal by selecting it with your mouse and pressing ctrl+shift+c, then open your editor of choice by typing (I choose nano here for example):

nano /etc/fstab

You can paste the lines after you cleared out the files' contents by pressing ctrl+shift+c. Now delete the lines marked with /dev/sda which is most likely your HDD and change the other two lines around so they finally look like this:

UUID=18055918-6045-42e9-9492-66e3fd29c199 swap swap defaults 0 0
UUID=69b5fb9c-774d-4d35-82f2-6a17faa446a1 / ext4 defaults 0 1

Then press ctrl+x to end your editing, choose y to save and hit enter

As next you need to fill your /etc/apt/sources.list, you can do so with copy and pasting the following block to your terminal and hit enter once.

cat > /etc/apt/sources.list << "EOF"
deb http://archive.ubuntu.com/ubuntu/ xenial main restricted
deb http://archive.ubuntu.com/ubuntu/ xenial-updates main restricted
deb http://archive.ubuntu.com/ubuntu/ xenial universe
deb http://archive.ubuntu.com/ubuntu/ xenial-updates universe
deb http://archive.ubuntu.com/ubuntu/ xenial multiverse
deb http://archive.ubuntu.com/ubuntu/ xenial-updates multiverse
deb http://archive.ubuntu.com/ubuntu/ xenial-backports main restricted universe multiverse
deb http://archive.canonical.com/ubuntu xenial partner
deb-src http://archive.canonical.com/ubuntu xenial partner
deb http://security.ubuntu.com/ubuntu xenial-security main restricted
deb http://security.ubuntu.com/ubuntu xenial-security universe
deb http://security.ubuntu.com/ubuntu xenial-security multiverse
EOF

As for the final configuration packages must be installed which require starting services via Upstart, these must be disabled in the chroot temporarily:

dpkg-divert --local --rename --add /sbin/initctl
ln -s /bin/true /sbin/initctl

Now run an update and you can install either ubuntu-server or ubuntu-desktop or even both as long you do server before desktop.

apt-get update
# Comment this out if ubuntu-desktop is not wanted
apt-get install ubuntu-desktop 
# remove comment if ubuntu-server is wanted
# apt-get install ubuntu-server

and finally add your user and put him into the proper groups:

adduser --home /home/<username> <username>

usermod -aG adm,cdrom,sudo,dip,plugdev,lpadmin,sambashare <username>

Where you can leave out sambashare if it gives you errors. Check if the group settings are correct by:

less /etc/group

You should see for each group listed in the above usermod command and entry like this (example sudo):

sudo:x:27:<username>

If not add the groups one by one with the above usermod command. Now it is time to maybe install drivers and useful tools, I list here only a few for your case necessary ones.

apt-get ecryptfs-utils testdisk schroot

To make GRUB only recognize your stick as OS and not adding the System you have build the stick on do:

echo 'GRUB_DISABLE_OS_PROBER="true"' >> /etc/default/grub
update-grub

Finalize the install by:

rm /sbin/initctl
dpkg-divert --local --rename --remove /sbin/initctl

Now you can reboot onto your USB and have a fully installed Ubuntu system to work with. If you run into the trouble of not being able to start the terminal which seems to be related to a bug in gnome-terminal you can check in TTY (ctrl+alt+f1 and login with your username and password) the content of /var/log/syslog if it contains lines like this:

May 7 22:30:00 NEXUS-TWO org.gnome.Terminal[1432]: Non UTF-8 locale (ISO-8859-1) is not supported!

To fix this it seem to be enough to execute sudo dpkg-reconfigure locales within TTY and choosing explicitly UTF-8. or set the language and so on within the GUIs settings. Afterwards reboot.

Answer 2

Here are some details. (Someone else should write a more independent review about the pros and cons.)

• Cloning: dd is a very powerful but also very dangerous tool, often nicknamed 'disk destroyer' or 'data destroyer'. dd is used to clone an iso file to a USB pendrive. When creating live-only USB boot drives, mkusb 'wraps a safety belt around dd'. This method works for all hybrid iso files, and most modern linux distros provide this kind of iso files. The new Ubuntu Startup Disk Creator in 16.04 LTS uses the cloning method too.

• Many other tools extract the content of the iso file to a FAT32 partition, for example Rufus and Unetbootin. (Rufus has also a cloning dd mode.)

• mkusb is a linux tool. It does not work in Windows.

  Prepare:

  sudo add-apt-repository ppa:mkusb/ppa
  sudo apt update
  

  Install mkusb:

  sudo apt install mkusb
  

• mkusb is a bash script, that uses several standard tools, available in most linux distros. mkusb uses zenity for a graphical user interface when available.

• mkusb-nox (No X) works in text mode, for example in Ubuntu Server. mkusb-dus (actually dus) can work in text mode too with dialog text mode menus or with a plain text interface. In a pure text mode system (for example Ubuntu Server) you should install the package dus and/or mkusb-nox, not the whole package mkusb,

  sudo apt install dus mkusb-nox
  

• Some of the other tools consist of compiled code.

• mkusb can create persistent live systems for Debian and Ubuntu. These systems are booted via grub2, and the iso file is cloned to a separate partition. A casper-rw partition will be created for persistence and a usbdata partition will be created for storing data and sharing data with computers running Windows. These persistent live systems can boot in UEFI and BIOS mode.

• mkusb-nox and mkusb-dus can create a USB installer for Windows 7-11 via extraction. It uses the grub-pc package to install grub2, which makes the USB pendrive boot in BIOS mode. This system can boot in UEFI and BIOS mode.

• mkusb does not create multi-boot USB drives. There are other tools for that purpose.

• mkusb can also wipe confusing data and/or restore a USB boot drive to a standard storage device with an MSDOS partition table and a FAT32 file system.

See the following links if you want more details,

• mkUSB-quick-start-manual
• mkusb-Persistent live systems

Answer 3

Wow that sounds complex, mkusb can do a Live Persistent install much faster than I can read either procedure above. They also sound like Full installs not Persistent Live installs. A persistent install uses a casper-rw file or partition. It is called persistent install because the word persistent in the boot file is used to activate persistence.

See: https://help.ubuntu.com/community/mkusb for details of mkusb.

A USB drive made with mkusb also has no problem doing a Full install to HDD or USB.