Broadly speaking, you need to install an EFI-mode boot loader for Linux on the USB drive you've prepared. This boot loader should be stored as
EFI/BOOT/bootx64.efi on the EFI System Partition (ESP), which is a FAT partition on the disk. There's a chance that the disk you've prepared lacks an ESP, so you may need to create it. (Even an ordinary FAT partition is usually sufficient, so if the disk has a FAT partition, you might not need a separate ESP.)
The tricky thing about this is that the boot loader you write to the disk must be properly configured to boot the installation on the disk. GRUB taken from the Ubuntu installation
.iso file won't work without changes, because it's configured to boot the Ubuntu installer, not the installation you've created on the USB drive. Thus, you must know enough about the boot loader to know how to configure it, and enough about your installation to know what values to put in the boot loader's configuration file.
Unfortunately, GRUB 2 is a very difficult boot loader to configure; its configuration file format is rather complex, and is usually created by specialized scripts. Even the binary may need to be modified for your specific situation, since the main configuration file (
grub.cfg) is normally located on the Ubuntu root (
/boot partition, meaning that the binary must be modified for your particular installation.
I've written a page on EFI boot loaders for Linux that covers many alternatives, most of which are easier to configure than GRUB 2. My own rEFInd is likely to be the easiest to set up and configure, since it tries to auto-detect Linux kernels when it starts, and it can usually figure out what options to pass to the kernel to get the system booted. Thus, configuration file changes needed to boot with rEFInd are minimal, and sometimes non-existent.
One big caveat in this is Secure Boot. If you want the computer to boot with Secure Boot active, you must provide either Shim (
shimx64.efi) or PreLoader (
preloader.efi) as the boot loader (renamed to
EFI/BOOT/bootx64.efi), with your follow-on boot loader renamed appropriately for Shim or PreLoader. Furthermore, depending on the boot loader in question, you might need to add keys or hashes to the firmware whenever you boot your disk on a new computer. In this case, it's probably worth going to the hassle of using GRUB 2 on the new computer, since Ubuntu provides all the pieces necessary to get a computer to boot Ubuntu without messing with keys or hashes.