Always backup your information before attempting any repairs.
Your assumption is correct. When you installed Ubuntu, it installed grub to the EFI partition of the internal HDD. In order to make the external drive bootable, it needs an EFI System Partition with grub installed in it. Usually this partition is the first partition on the drive. However, moving the root partition can cause boot errors.
If reinstalling is a viable option
Repartition the external hard drive. Delete the linux partition, then make a 500MB partition formated FAT32 and Flagged as ESP (Efi System Partition). Use the remaining space for the linux partition formated Ext4. The result should be:
- EFI partition 500MB formatted FAT32 and flagged ESP
- Linux partition formatted Ext4 no flags
- Swap partition formated swap no flags
- NTFS partition
Then reinstall Ubuntu. Use the manual partitioning option to select your precreated partitions. Make sure the EFI partition of the internal drive is set to "Do Not Use".
Booting an external drive requires the use of the default media path. Boot the LiveCD, and mount the EFI partition on the external (you may need to teporarily remove the ESP flag to do this). Copy
EFI/boot then rename it
bootx64.efi. Unmount the EFI partition and set the ESP flag if you had to remove it.
Attempt to repair the current installation
Boot from a LiveCD, and use
gparted to shrink the linux partition by 500MB. Create a new partition with the freespace, format it FAT32, and flag it ESP. Record the UUID of the new partition.
Mount your linux partition. Edit
etc/fstab. Look for a line similar to:
# /boot/efi was on /dev/sda1 during installation
UUID=BB1E-83B5 /boot/efi vfat umask=0077 0 0
Replace the UUID with the one for the new EFI partition.
Next, mount the EFI partitions for both drives (you may need to remove the ESP flag temporarily). Copy
/efi/ubuntu/shimx64.efi from the internal drive to
/efi/boot/ on the external then rename it to
bootx64.efi. Note: you will have to make the folders on the external drives.
After either repair
Finally, mount the EFI partition of the internal drive. Delete the folder
/EFI/ubuntu. Reboot and open system settings by pressing the appropriate function key while booting. Change the boot order so USB HDD is before the internal drive.
You should now boot linux when the external is connected, and Windows when the external isn't connected.