I actually just now helped one friend over skype to do that with Lubuntu and ive put for him 2y ago one persistant xUbuntu Live USB.
U can make it to be as Live so u can still use it as installation but with persistance witch means it saves files to persistance file witch cant be seenable from windows. Also Live means it will not change thouse 700mb of usb stick even after updates to remain original and updates will be using persitance file part.
OR u can install ubuntu on USB but then in partitioning step use manual and use all usb as system / partition and dont put swap couse swap will be automatically put in automatic installation but u dont need usb to be all the time using swap and u dont need hibernation on usb right?
So 1st step
From computer .. from what else.. go to http://www.ubuntu.com/download/ubuntu/download and download an actual Ubuntu image and i recommend better using 10.04 LTS couse thats lighter and faster and more understandable or even better i recommend 10.04.03 netbook remix couse thats even more faster.
==PRepearing Live USB with installation possibility and persistance file==
If ull use Ubuntu to make USB then System > Administration > Startup Disk Creator
Browse to desired downloaded Ubuntu file and if want u can erase USB from Startup Disk Creator and choose Persitance option and slide to use as much as want.
Press make and in process enter 3 times password.
==Installing on USB==
If u dont need option to be able to later infect some other computers with ubuntu then u can dont need live usb but u can just install Ubuntu on usb. To do that u can use LiveCD/USB and install on USB and dont put swap partition on it. Since choosing automatically to install Ubuntu on chosen partition will make swap file u need to choose Manual installation and format whole USB to be just partition / and done. It will warn that u didnt put swap but since its waste of space and might not anyways be able to work as much as fast possible if swap is beeing used and that means written to USB swap partition as virtual memory witch slows work more then we dont need it.
theres also methods in https://wiki.ubuntu.com/LiveUsbPendrivePersistent