Persistent live system
UNetbootin has a built in persistence option, it uses a file named casper-rw for persistence. Maximum size of UNetbootin's persistence file is 4GB and is size limited by the FAT32 file system.
Mkusb creates an Ubuntu flash drive that boots using grub2. It uses a FAT32 boot partition, a ISO9660 read only OS partition, an ext casper-rw persistence partition, (not limited to 4GB), and a NTFS partition that can be used by Linux or Windows.
Full installed system
You can alternately do a Full install to USB drive.
It is recommended to first unplug the internal HDD, boot the installer drive, insert the USB target drive and install to it.
"Something else" can be used to add a boot partition, a NTFS partition for Windows, a "/home" partition and swap.