I've tried to install Ubuntu 12.04 from USB following the official instructions I created the USB key and started the install.
I chose to install Ubuntu alongside Windows and kept the suggested partitioning scheme.
At the end of the install I rebooted and the bootloader is not there and Windows boots.
I have tried with both Universal USB Installer (suggested on the Ubuntu site) and with Unetbootin.