OP asks for "without touching my existing Ubuntu install." My answer describes how to get this working, with a real install, not a persistent live USB install. Persistent installs are pretty fragile, and updating the kernel is hard. A real install is better.
If you want to boot from your external device, not just install Ubuntu on it, you need to work around an installer bug which rewrites your boot partition on your internal drive and does not install an EFI boot partition on your external device, no matter what you tell the installer. If you don't do this, you will still end up with an install on your target usb stick, but it will only work on this computer, because it won't have its own boot partition.
The EFI partition is part of the drive that the BIOS looks for when it is starting. If a drive has no EFI partition, it can't be a boot drive. If a USB stick or external drive does not have an EFI partition, you can still install Ubuntu to it, but it will need a "bootable" drive to actually launch. So if you want a USB stick or external drive that you can take from computer to computer and boot independently, that drive must have its own EFI partition.
Here's what works for me in Ubuntu as recently as 21.10.
Tested on various laptops. I have turned off legacy boot. UEFI is 100% in use (this will be the default setting on anything sold in the last five years).
I turn off secure boot in BIOS.
Installing onto a second drive is a pain because the ubuntu installer uses the first EFI partition it sees, which is the one on the internal hard drive, regardless of any attempt you make to specify an alternative location for the EFI partition. So when you try to set the bootload device to your target USB drive, you are ignored. It's a fairly old bug, but as of 21.10 release, still present.
The bug means you won't get an EFI partition on your USB stick even when you asked for it, so you can't boot from it.
https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1396379
Summary: To workaround it, disable the internal EFI partition by using gparted to edit its flags immediately before beginning your install. Then the installer won't find it, and the bug is not tripped. Later, re-enable the flags. This is a trivial step. It is almost the logical equivalent of physically disconnecting the internal drive, which for sure also works around the installer bug.
The steps I took:
Before you start the install:
You'll need a standard ubuntu live USB device, and a target usb stick to install to.
Boot into Ubuntu live USB in the "try first" mode.
Using gparted (you may have to install it first, sometimes Ubuntu doesn't include it on the live disk,): ...
re-partition your target external drive with a GPT partition table. GPT partition tables are needed for a UEFI (modern) bootable drive.
Make a 500MB partition type FAT32. You may as well also set up the desired partition(s) for your Ubuntu install. You may find it handy to label the desired / partition because when you install you will have three drives: your internal drive, the live image installer drive, and your target drive.
After applying those updates, change the flags on the small 500MB partition you just created. Right click on the small partition, and Manage Flags. These changes are actioned immediately (but note, you must actually create the partition first by completing the previous step)
Tick to turn on boot, esp and hidden.
Install
You have booted with a live-disk USB image, as per a normal ubuntu install. So you have two USB devices: your target device, and the live-disk USB drive.
Edit the EFI partition flags on your internal drive and untick those same three flags that you set on the target device EFI partition. This will stop the Ubuntu installer for using it as the boot partition.
Here is a short video doing the flag editing in gparted: https://youtu.be/sdgrmylH6pc
Now, when you install, the installer will see only one EFI partition, on your target device. This is the novel step which I haven't seen documented elsewhere.
Begin an ubuntu install. Proceed until you see the disk setup tab of the installer. You want the fully manual approach of course, "Something else" on the partitioning stage.
You specify the way the boot loader is installed when doing the install. If your target drive is mounted as sdc and the EFI partition you made is therefore sdc1 (the first partition), then you will be installing the boot loader onto device sdc, and the EFI partition will be sdc1.
Scroll to find that partition. It should say "efi" in the Type column. Click "change" to be sure: The installer should say "Use as: EFI System Partition". You won't actually be changing anything. No need to format it.
As you scroll through the partitions, review the Type column. There should be no EFI partition on your internal drive, since you turned off the partition flags on your internal drive EFI partition. Of course, the partition still shows up as a FAT32 partition. That's ok.
You will also see the EFI partition of the live disk you booted from to do the install, that's ok. The installer is smart enough to ignore that.
Choose your desired target partition for / (sdc2, perhaps,... whatever you already made above) and do a normal Ubuntu install.
After Install
Restore flags on your internal EFI partition
After the install, reboot to the new installation on the USB stick. You will need to use your BIOS "select boot device" option because the computer needs to use the boot partition you just made, which it has never seen before. On my Thinkpads, F12 is the shortcut to this part of the BIOS menu.
You should see several choices of boot drive in the boot menu, and one of them is the external drive. Some bios menus show the default label as 'ubuntu' so it's a bit confusing to see it more than once. Sometimes changing the boot device causes the BIOS boot to restart (it does on my Thinkpad), it looks like something bad happened, but it's ok.
Later when you boot without your USB stick, the bios should be smart enough to revert to the last known good EFI device (your internal device), but you may need to reselect an EFI boot choice manually.
Tip: How to re-label the USB boot entry to avoid duplicate 'ubuntu' entries
If you get duplicate EFI boot options labelled ubuntu, you can fix it.
Relabelling EFI menu options is very handy, but a bit tricky. Do this once you are booting ok from your new install. This step is optional.
Make sure you boot into the installation on your external drive, then
sudo efibootmgr -v
You are booted from the first row in the list.
Note the name of the file used to boot, and note the number of the partition.
my output for the first entry is:
HD(1,GPT,...) .... File(\EFI\ubuntu\shimx64.efi)
and then have a look at this thread: https://www.kubuntuforums.net/showthread.php/68851-Labels-on-UEFI-Boot-Entries-using-efibootmgr-L
I did this to relabel mine 'owcUbuntu':
efibootmgr -c -d /dev/sdb -p 1 -L owcUbuntu -l \\EFI\\ubuntu\\shimx64.efi
knowing that the boot drive in my case is sdb and since I made the EFI partition first, the value of the -p argument is 1. Note: please check what your actual boot disk is :) use gparted or df
USB 3 Recommendation: I have tried this on a range of USB 3 sticks. The best experience by far (very far) has been the Samsung USB 3 "Bar" sticks. They are really fast (for USB 3 sticks) in this use-case (random access, ext4 partitions with journalling) and quite robust.