First I must warn you: on the average thumb drive, Ubuntu will run VERY SLOWLY the way you wish to install it. It is also likely that you will run out of space very quickly (I happen to have installed and attempted to use Ubuntu in this very same way once).
Ok, so the first thing you need is two thumb drives (yes, two): one to run as a live usb and one to install onto (this one will need to be at least 5 GB).
So first, create an Ubuntu live usb on the first flash drive. Plug both flash drives into your computer and boot to the one that is the live usb (if you can't determine which that is, pull out the one that isn't and then plug it back in once you have booted).
Once the live usb boots, click "Install Ubuntu." On the screen asking you to choose to overwrite your current os, install Ubuntu alongside it, or something else, choose something else. This will bring you to a tool which you can use to dedicate partitions to specific purposes in Ubuntu.
Your hard drive is shown automatically. First make sure that none of these partitions are marked "swap." This happened to me and I lost the swap on my regular Ubuntu installation. Select your thumb drive that you would like to install to from the devices menu. There should be just one big partition on it. Shrink this partition by no more than 1 GB. Then, mark the big partition as the root partition and the small partition as the swap partition.
With that done, click continue or next or whatever the button is (sorry, I'm not looking at it right now) and continue with the installation as you would normally.
The installer will do the rest of the work after you input just a little more information. It will install Ubuntu onto your flash drive.
However, as I stated above, this will run very slowly and run out of space quickly, no matter how good the computer you are using it on is. I recommend simply making this flash drive into an Ubuntu live usb with the biggest persistence file possible, you should be able to run fine off of that, but a flash drive is still slower than a hard drive.
I also must include one more warning: this means that your installation is potentially going to be used on multiple machines, so if one specific machine needs something else to run (I am thinking of drivers) Ubuntu, it may render the installation unusable for any machine except the one you installed drivers for.