What you need
- An empty external (USB) hard drive. I used a USB 3 drive enclosure with a cheap 256Gb SSD. For those of you worrying about performance,
the USB 3.0 port is faster than the maximum transfer rate of any SSD
I've seen to this date.
- Ubuntu Linux bootable USB drive. Very easy to create using Rufus on Windows. I used a cheap, promotional flash drive. Reduce, reuse,
recycle FTW.
- Windows System Repair Disc (a bootable USB drive with Windows recovery tools which you can make yourself) or a Windows installation
or rescue CD-ROM / DVD-ROM. I used another promotional flash drive.
Remember to take backups of all your critical files and folders before
continuing. You are going to make changes to your computer. If it
breaks don't cry, I warned you. Also test both your bootable media.
Twice. You will need them both!
Caveats
Having a dual- or multi-boot system isn't as straightforward as it
used to be a decade ago. Nowadays, in the interest of security,
performance and backwards compatibility, there are many semi-hidden
options and features which can get in your way. Given enough
experience and patience you can work around them. Below is a selection
of problems that got me moderately stumped along the way.
Secure boot caveat
I have only tested these instructions with Secure Boot turned off.
Even though Ubuntu does support Secure Boot (it comes with signed
bootloaders) I have no idea if my method uses the signed bootloaders
or not. I suggest turning off Secure Boot if possible.
FastBoot caveats
Many boards come with some sort of "fast boot" or "boot optimization"
options. For example, my Intel NUC has an option called Fast Boot
which won't let me choose an alternative boot device at startup. It
also has an option to support Intel Rapid Start Technology which does
get in the way of booting to multiple OS. I had to disable both.
It's worth noting that Windows 8 and 10 have an Fast Startup or Fast
Boot feature. This feature takes some shortcuts when it comes to
booting and also makes the NTFS filesystem remain in a dirty state,
making it unwriteable from Linux. It's best to understand what it does
and disable it if you plan on writing to your Windows drive from
Linux.
USB host controller caveat
Some firmwares will present the USB host controller as UHCI (USB 1.1)
at boot time. When Linux probes for an xHCI (USB 3) host controller
during the boot process they will respond positively. At this time
Linux loads the xHCI driver and the USB host controller resets itself.
However, your root filesystem is inside a device attached to this USB
controller. Therefore the controller resetting means that Linux can no
longer communicate with the USB-attached hard drive. Therefore the
Linux boot will hang forever without any further indication as to what
went wrong.
Most affected boards (including my Intel NUC) have an option to enable
the xHCI host controller interface by default. Enabling the xHCI
option in the BIOS fixes the hanging boot issue. If you are only using
modern operating systems with USB 3 support (anything newer than and
including Windows 8.1 and Ubuntu Linux 15.04) you can safely enable
that option.
Installing Linux
Boot your computer from the Ubuntu Linux bootable USB drive. Remember
that you may have to enter your computer's boot manager to do that (on
my Intel NUC I have to press F10; on most other BIOS I've seen it's
F9; consult your BIOS documentation).
Install Ubuntu Linux regularly. When it prompts you about the disk
layout choose Something Else and partition your external HDD the way
you want. I chose to create a modestly sized root partition (about
40Gb), a swap partition that's as big as the biggest RAM configuration
I am going to be using this installation with plus one Gb (my
computers max out at 16Gb so I made a 17Gb swap partition) and the
rest of the disk went to a massive /home partition.
Caveat: I chose to use btrfs which – as I learned along the way – makes things a bit more complicated down the line. For your sanity's
sake I recommend using ext4. This guide assumes the use of btrfs and
will point out the caveats with this approach.
Fix Windows 10 boot
Unfortunately the Ubuntu Installer assumes that you want a dual
booting configuration alongside Windows. Therefore it adds itself
(actually, the bootloader it uses, GRUB2) to your computer's UEFI
configuration. This causes two problems. For starters, the external
HDD is not portable as you cannot boot with it on another computer.
Secondly, if you remove this external HDD your Windows won't boot.
Bummer. We have to fix that.
- Shut down your computer.
- Disconnect the Ubuntu HDD
- Boot from the Windows system repair disc USB drive (or a Windows installation or rescue CD-ROM/DVD-ROM).
- Select Repair your computer.
- Select the operating system and click Next.
- Choose Command Prompt.
In the command prompt run
diskpart
sel disk 0
list vol
Verify that the EFI partition is using the FAT32 file system. It will have a volume ID, let's say 99. Now we need to assign a drive
letter to it. Back in the command prompt type:
sel vol 99
assign letter=z:
exit
Now we need to fix the boot record. Again in the command prompt type:
z:
cd EFI/Microsoft/Boot
bootrec /FixBoot
Finally, we need to re-create the BCD store which tells the Microsoft boot loader where to find Windows so it can boot it. From
our trusted command prompt:
ren BCD BCD.old
bcdboot c:\Windows /l en-us /s z: All
If this didn't work try:
ren BCD BCD.old
bootrec /RebuildBcd
At this point exit the command prompt and shut down your computer.
Create an ESP on the Ubuntu HDD
A hard drive is not bootable with UEFI unless it has an ESP (EFI
System Partition). An ESP is simply a FAT32 partition with a special
flag that tells the EFI BIOS to look inside it for boot information.
We have to create one on your hard drive.
- Plug in your external HDD and the Ubuntu Linux bootable USB stick.
- Boot with the Ubuntu Linux bootable USB stick using the option to try Ubuntu before installing.
- Open a Terminal (Ctrl+Alt+T)
- Run
sudo fdisk -l
to get a list of partitions.
- Identify from them the drive that has the Linux partitions, in my case
/dev/sdb
. I'll call it /dev/sd<b>X</b>
from now on.
- Also identify the partition that contains the root filesystem. I will call it /dev/sdXY from now on.
Launch GParted from the Terminal:
sudo gparted /dev/sdX
Why not just click on GParted on your desktop? Well, I kept receiving errors about the Ubuntu Linux bootable USB stick because it
was already in use. Of course it is, I am using it to run the computer
off it, duh!
Resize the first partition on disk to have another 200 Mb of free space after it.
- Create a new partition on the free space, changing the file system to fat32.
- Apply operations. You need to do that now for the next step to be possible.
- Right click the new partition.
- Click on Manage Flags.
- Set the boot and esp flags. This is what makes the partition "special" to the EFI BIOS.
- One more thing! Note down the the partition that contains the ESP filesystem. I will call it /dev/sdXZ from now on.
Make sure the Ubuntu installation on the external HDD can see the ESP
The new ESP on the external drive must be visible by the Ubuntu
installation in the HDD. Otherwise GRUB2, the Linux bootloader, won't
be able to update itself, making your system unbootable after the next
kernel update at the latest.
Launch GParted from the Terminal, as we saw above:
sudo gparted /dev/sdX
Double-click the partition with your Linux root (/) filesystem on the external HDD
- Note down the UUID, e.g.
01234567-89ab-cdef-0123-4567890abcde
- Double-click the new FAT32 partition and note down the UUID, e.g.
0123-ABCD
- Close GParted
- Open a Terminal
The process is different depending on the format of your root
partition on the external hard disk.
If you DID NOT use btrfs (e.g. you used ext4)
sudo umount /media/ubuntu/01234567-89ab-cdef-0123-4567890abcde
sudo mount /dev/sdXY /mnt
If you DID use btrfs
If you DID use btrfs, you made your life complicated. We need to mount
the btrfs subvolume containing the root partition instead of the
entire partition. Otherwise you'll never be able to install GRUB and
you'll probably lose an entire day, like me.
btrfs subvolume list /media/ubuntu/01234567-89ab-cdef-0123-4567890abcde
This will give you a line with a numeric ID. Let's say 123. Note it
down.
umount /media/ubuntu/01234567-89ab-cdef-0123-4567890abcde
mount /dev/sdXY -o subvolid=123 /mnt
The rest of the instructions are common, no matter if used btrfs, ext4
or something else
sudo nano /mnt/etc/fstab
Install GRUB2 on the external drive's EFI System Partition
Right now our external drive has an empty ESP. We need to put a
bootloader in it to make it actually, well, bootable.
First caveat: all the instructions you find on-line assume you are using a dual boot system with Windows or macOS. When you have an
external drive it is critical that you use the --removable
option in
the last step. This installs the EFI bootloader under the special
"fallback path" EFI\Boot\bootx64.efi in the ESP. Normally this not
supposed to be used for permanently installed Operating Systems. It's
the mechanism used by EFI BIOS to boot arbitrary external media.
Technically, that's exactly what our external hard drive is: arbitrary
external media!
Second caveat: installing the bootloader is only possible from inside the Linux installation we want to boot. However, we need the
bootloader to boot that installation, leading to a Catch-22 issue. The
solution is to run the bootloader installation through a chroot jail.
The actual caveat that got me stumped for a day comes from the fact
that I am using btrfs (because it's so much better for SSDs!). btrfs
has subvolumes. If you mount the entire partition instead of a
subvolume the grub-install script can't figure out the mapping between
paths and devices, therefore failing to install itself on the ESP,
returning the cryptic error:
/usr/sbin/grub-probe: error: cannot find a device for / (is /dev mounted?).
The error is misleading! /dev
is mounted if you follow my
instructions below. The actual problem, as I understand it, is that
there is a discrepancy between the mounted device and the path to the
chroot root. That's why I had you mount only the subvolume containing
the root filesystem in the steps above. If you were not paying
attention, you are not following the instructions step-by-step, you
rebooted before this step or just came here directly looking for a
solution to your problem about GRUB not installing look above for
instructions on mounting the correct btrfs subvolume.
We need to prepare the chroot environment. The ESP must be mounted in the correct place and we have to bind system mount point for some
special trees (most notably /dev). Moreover, we will copy the
resolv.conf file to let the chroot environment have network access
should it need it.
mount /dev/sdXZ /mnt/boot/efi
for i in /dev /dev/pts /proc /sys; do sudo mount -B $i /mnt/$i; done
cp /etc/resolv.conf /mnt/etc/
modprobe efivars
Finally we enter the chroot environment and install Grub in a way suitable for a removable device (see the first caveat above).
sudo chroot /mnt
grub-install -d /usr/lib/grub/x86_64-efi --efi-directory=/boot/efi/ --removable /dev/sdX
Now your external HDD is bootable. Reboot your computer, select it
from the boot media selection of your UEFI BIOS and you're done!