I have tried a persistent installation of Ubuntu 14.10 on my PNY 32GB USB2.0 to use as a portable OS to boot my PCs. Everything is alright. I can boot my two PCs with the flash drive, however, as expected, it runs very slowly. Also 32GB is not enough. Therefore, I bought a Sandisk 128GB USB 3.0 and installed Ubuntu on it. I did everything exactly the same. However, my PCs won't boot from USB now. I have the correct BIOS setting to boot from USB HDD first, but the bootloader on the USB does not run, and the bootloader on the internal hard drive is run instead. The Ubuntu in the flash drive should be OK, because if I add it to the internal hard drive bootloader, it can run properly. So the Ubuntu itself should be OK. Only that bootloader on it will not run.
I think I found the answer. When I installed on USB 2.0, I booted the installation CD in legacy mode. When I installed on USB 3.0, I booted it in UEFI mode. So the problem is not USB 2.0 or 3.0, but legacy or UEFI. I've done reinstallation on USB 3.0 with the installation CD booted in legacy mode, and now I can use the USB 3.0 pendrive to boot my PCs. However, it can only boot in legacy mode. Some actions are not allowed, e.g. boot-repair or os-uninstall. So my question now becomes how to install Ubuntu 14.10 on a pendrive so that it can boot in UEFI mode?