Recent changes to Ubuntu (13.04 and beyond) and Windows (8 for sure and beyond) start using GPT partition tables and UEFI booting by default to piss people off that haven't experienced them before.
You are likely running into a legacy vs. UEFI boot problem or your HDD boot order hasn't been arranged after you plugged in the boot disk.
Plug in the USB or DVD before getting to the BIOS, go to the boot settings and move it to the top of the boot order.
If it's the former issue:
Go into your BIOS and select whichever mode (legacy or UEFI) is not currently active to troubleshoot. It may even be a combination of both of these solutions above.
This will not work with an installation of Windows 8 already on the PC using UEFI/Secure boot unless you use legacy mode and dual-boot in an annoyingly complicated way using EasyBCD
I have run into this issue many times and have completely given up on EFI booting since it makes no difference to me how fast the PC loads. That just makes getting to the BIOS 102031x harder.
Besides, who uses Windows anymore anyway? Right? ;) time to go *nix only son. Join the cool kids.
This may not work for sure in your situation but hopefully it helps.
Let me know if there's more I can do.