How do I install Ubuntu to a USB key?
Is it possible to install Ubuntu 11.04 to a 4gb USB key and run the OS but when you shut off for it to retain files, settings, etc?
Absolutely. A USB key acts just like any HD. Most BIOSes these days can boot off them.
I ran my NAS off a USB stick for about a year to keep the power usage down. It's deathly slow however, you will frequently be twiddling your fingers waiting for it to write a file, and you can generally go and make a three course dinner while it does an apt-get dist-upgrade!
There is nothing special you need to do to make it retain settings, you write to it like any other normal filesystem once its installed.