Have a look at this guide and video on my website on installing Ubuntu to a USB drive. It will help you do exactly what you want and it's very simple.
How to Install Ubuntu To USB Drives
Installing Ubuntu to an external hard drive or USB memory stick is a very safe way to install Ubuntu. If you are worried about changes being made to your computer, this is the method for you. Your computer will remain unchanged and without the Usb inserted, it will load your operating system as normal. When you connect and boot from the USB drive you will be given the choice to load Ubuntu or your usual operating system.
What we need to install Ubuntu to a USB drive is a computer, an Ubuntu live CD/USB, and a USB drive. 8 GB is the minimum recommended size for a functional and useable system (although 4 GB is the minimum). We recommend an external hard disk and at least 20 GB.
It is recommended to partition your USB drive, but not necessary, assuming you have 2GB RAM or more. Partitioning can be done from the Ubuntu live CD/DVD using 'disk utility', or from the installation partitioning menu.
We recommend using a Live CD/DVD and unplugging any other USB drives as this makes life easier. We will assume that you are using an unpartitioned USB drive and CD/DVD for this guide.
How To Install Ubuntu To A USB Drive
Insert the Ubuntu Live CD/DVD, switch on the computer and tell it to boot from CD/DVD using your 'BIOS'. It will take a couple of minutes to load and you will be presented with two choices. 'Try Ubuntu' or 'Install Ubuntu', you should select 'Install Ubuntu'
You will then be presented with a number of options. You need to select the bottom option 'Something Else'.
This will bring you to the partitioning menu. Your Primary hard drive will be listed as 'Sda' followed by any partitions that are on it like Sda1 or Sda2. Below this will be your usb drive, it will be listed as 'Sdb'. Click on 'Sdb1' which is the only partition on the drive and select 'change'.
You need to select 'use this partition as Ext4 File System' (some distributions such as Mint reqiure you to manually select 'format' at this point). Then you need to set the 'mount point' as '/' which is the root file system and click OK. You will be taken back to the previous menu and that partition will have a tick next to it. Now click on 'Sdb' just above that partition, this selects it as the device to install to.
Now just below is an option for where the bootloader is to be installed. It is VERY IMPORTANT that you CHANGE THIS OPTION. THE BOOTLOADER MUST BE INSTALLED TO YOUR DEVICE listed as '/dev/Sdb'. If you do not do this the bootloader will be installed to your internal drive. You are now ready to install to your external device, simply click 'Install'.
You will need to answer a few simple questions like 'name' and 'create password', then you can sit back and relax.
You will need to tell your BIOS to boot from the USB device each time you want to use it.
You can easily set USB as your first boot device in the bios, and your normal system will load if the device is not connected. Do not be surprised if your Usb installation takes 3 or 4 minutes to fully boot, especially when using a cheap Usb memory stick. If you choose to install from a usb drive instead of CD/DVD, or you have multiple hard drives and Usb drives connected you need to make sure you are installing to the correct device, as it may not be sdb.
The easiest way to ensure that you are installing to the correct device is to use disk utility. Open disk utility before connecting the device and note the devices on the left. Then connect your device and it should appear at the bottom of the list. Click on that device and look at the top right of the window for 'Device'.
It will say something like 'Device : dev/sdc' and so 'sdc' would be the device you need to install to.