In short, I plan to use the Ubuntu Live to develop for my Raspberry Pi. This include remote ssh, cross-compiling, etc. I do not wish to install Ubuntu, but I can use it live. Is there any difference for my purposes between live and installed Ubuntu? I understand I cannot store any files, but that's fine.
Is there any difference for my purposes between live and installed Ubuntu?
Not much. You're going to have lower performance because you're running off a USB stick or DVD and so it'll take longer to load things. You'll also not have as much RAM available because you've got much of the system loaded into RAM.
If you're trying to compile software & other CPU intensive things I wouldn't suggest doing them on the LiveUSB/LiveDVD unless you've got 4GB or more of RAM.
I understand I cannot store any files, but that's fine.
Actually, you can! If you use UNetBootin (which can be downloaded & run on all platforms (Windows, OS X, & Linux) you can tell it to create space on the USB stick to store changes between reboots.
Granted, you'll need something larger than 4GB (8GB+), but you can actually store stuff between LiveUSB boots. Note, this is only for Ubuntu distros:
You can also do something similar in Startup Disk Creator, but you'd need another USB stick to install the ISO on, since you obviously can't write the ISO to the USB stick you're running off of: