A USB device can be presented in the list of boot media as a Removable Device, but it can also be detected as one of your Hard Disks. In this case you won't see it on the list (your HDD is going to be the default). In this case (possibly your case) you need to get into BIOS by pressing F2 or Delete during boot -> Find Boot tab / Boot Device Priority section there -> Select your USB drive as the first hard disk and your primary HDD as the second -> Save and Exit. You should now be able to boot from your USB drive by selecting hard drive option from the list of boot media.
Afterwards, when you remove the USB drive, you should be able to boot from HDD without any other changes, but can return your HDD to the first bootable hard disk to speed up the process.
Some BIOS allow for choosing the boot device at boot time (often with F12). See your manual for details pertaining to your particular make and model.
Choosing the boot device by this method removes the necessity of changing boot priority in your BIOS.
Citation: Vast majority of answer blatantly copied from a closed question answer by Pavel A in the 2nd source (1st source listed also cites 2nd source)
my computer does'nt recognize the usb when booting